How To Group Test Cases In TestNG [with Examples]

TestNG is an open-source automation testing framework inspired by JUnit and NUnit. The framework supports data-driven testing, parallel test execution, testing integrated classes, provides access to HTML reports, amongst others. TestNG can be seamlessly integrated with Jenkins, Eclipse, IntelliJ IDEA, Maven, etc.

Annotations in TestNG are useful for controlling the test methods’ execution flow and running multiple tests in Parallel. Parameterized TestNG tests with Selenium WebDriver are handy for running test(s) against different browser and platform combinations. As a part of our TestNG tutorial series, we look at how to group test cases in TestNG. The usage of groups in TestNG is demonstrated with relevant TestNG group examples.

What Are TestNG Groups?

As the name indicates, grouping in TestNG lets you group multiple test methods in named groups. Through groups in TestNG, you have the provision to run a particular set of tests that belong to a group or multiple groups.

Consider a scenario where the test suite comprises different test types, e.g., unit test, integration test, smoke test, etc. Instead of adding all the test scenarios in a single test suite (or multiple test suites), you can perform ‘grouping’ test methods. This helps in segregating the test scenarios into different test categories – test type (e.g., smoke, functional, etc.), functionality being tested (login, checkout, etc.), and test combinations (e.g., browser set to Chrome, Firefox on Windows 10, etc.).

The sophisticated groupings of test methods through TestNG groups are used for:

• Declaring methods that belong to groups.
• Specifying groups that contain other groups.
• Running tests within the same group or multiple groups.
• Including (or excluding) tests in a group.
• Including a certain set of groups (or regular expressions) while excluding another group
• Running tests that depend on the other tests and avoiding unnecessary test execution if the dependent test has failed.

Advantages of TestNG Groups

Here are some of the major advantages of groups in TestNG:

• Flexibility to partition tests, thereby reducing the efforts involved in the maintenance of the tests.
• Avoids the need to recompile the code if two different sets of tests are run back to back.

TestNG groups can also be used for running tests in Parallel at the ‘Test’ level or ‘Method’ level. In further sections of this ‘How to group test cases in TestNG’ tutorial, we would demonstrate parallel testing in TestNG and Selenium using Test Groups.

How to group test cases in TestNG?

Groups in TestNG are specified in testng.xml under the <suite> or <test> tag. Groups under the <suite> tag apply to all the tests included under the <test> tag in that particular <suite>.

To group tests in the source code, you have to use the @groups attribute of the @test annotation. TestNG provides the option to structure the test such that the entire test class belongs to a particular TestNG group or a couple of methods belong to the TestNG group. Also, test methods can belong to one or more TestNG groups.

Let’s take a simple example where the test code has three test methods (test_method1, test_method2, and test_method3). We create two TestNG groups – group1 and group2.

Here is how to group test cases in TestNG. As shown below, test_method1() is part of both the groups and test_method2() & test_method3() are a part of the group2 & group1 respectively.

public class Test1 
{
  @Test(groups = { "group1", "group2" })
  public void test_method1()
  {
    //Test implementation
  }

  @Test(groups = {"group2"} )
  public void test_method2()
  {
    //Test implementation
  }

  @Test(groups = {"group1"})
  public void test_method3()
  {
    //Test implementation
  }
}

The TestNG groups are invoked in testng.xml:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="TestNG Group Test">
    <test verbose="2" preserve-order="true">
        <groups>
            <run>
              <include name = "group1"></include>
              <include name = "group2"></include>
            </run>
        </groups>
        <classes>
            <class name="example1.Test1"></class>
        </classes>
    </test>
</suite>

As seen in testng.xml, the groups are included under the <group> tag. As we want to run the tests included under both the groups (group1 and group2), we include the required group names under the <run> tag.

<groups>
   <run>
    <include name = "group1"></include>
    <include name = "group2"></include>
   </run>
</groups>

Demonstration: How to use TestNG Groups

To demonstrate running tests within the same groups, within multiple groups, including (and excluding) test methods, etc., we use the cross browser testing scenarios that will be executed on the cloud-based Selenium Grid by LambdaTest.

• Test Group – 1 (GroupName – Search)

Test Scenario – 1 [OS – Windows 10, Browser – Chrome, Version – Latest]

1. Go to Google.
2. Enter the search string as ‘LambdaTest.’
3. Assert if the page title does not match the expected title.

Test Scenario – 2 [OS – MacOS Catalina, Browser – Safari, Version – 13.0]

1. Go to Bing.
2. Enter the search string as ‘LambdaTest Blog.’
3. Assert if the page title does not match the expected title.

• Test Group – 2 (GroupName – ToDo)

Test Scenario – 1 [OS – Windows 10, Browser – Firefox, Version – 68.0]

1. Go to LambdaTest ToDo App.
2. Mark the First Item (li1) as Done.
3. Mark the Second Item (li2) as Done.

Test Scenario – 2 [OS – MacOS Catalina, Browser – Safari, Version – 13.0]

1. Go to LambdaTest ToDo App.
2. Mark the First Item (li1) as Done.
3. Mark the Second Item (li2) as Done.
4. Mark the Third Item (li3) as Done.

Here is the overall project structure which we created in IntelliJ IDEA IDE:

In the TestNGGroups project, we create a package named org.testnggroup, where we create the following class files (that contain the implementation of the relevant test scenarios):

(Implementation of Test Scenarios in ‘Test Group 1’)

package org.testnggroup;
import org.openqa.selenium.*;
import org.testng.Assert;
import org.testng.IExecutionListener;
import org.testng.annotations.Test;
import java.net.MalformedURLException;

public class TestNG_SearchGroup extends Helper implements IExecutionListener
{
    public static String status = "passed";

    @Override
    public void onExecutionStart()
    {
        System.out.println("onExecutionStart");
    }

   @Test (priority = 1, groups = { "Search" })
    public void test_GoogleSearch() throws InterruptedException, MalformedURLException
    {
        String search_string =" LambdaTest";
        String exp_title = "Most Powerful Cross Browser Testing Tool Online | LambdaTest";

        setupThread("test_GoogleSearch", "test_GoogleSearch",
                "Windows 10", "Chrome", "latest");
        WebDriver webdriver = getDriver();
        webdriver.navigate().to("https://www.google.com");
        webdriver.manage().window().maximize();
        System.out.println("Started session");

        try {
            /* Enter the search term in the Google Search Box */
            WebElement search_box = webdriver.findElement(By.xpath("//input[@name='q']"));
            search_box.sendKeys(search_string);

            search_box.submit();
            Thread.sleep(3000);

            /* Click on the first result which will open up the LambdaTest homepage */
            WebElement lt_link = webdriver.findElement(By.xpath("//span[.='LambdaTest: Most Powerful Cross Browser Testing Tool Online']"));
            lt_link.click();
            Thread.sleep(5000);

            String curr_window_title = webdriver.getTitle();
            Assert.assertEquals(curr_window_title, exp_title);
            status = "passed";
        } catch (Exception e) {
            System.out.println(e.getMessage());
        }
        if (webdriver != null)
        {
            ((JavascriptExecutor) webdriver).executeScript("lambda-status="+status+"");
            webdriver.quit();
        }
    }

    @Test (priority = 2, groups = { "Search" })
    public void test_BingSearch() throws InterruptedException, MalformedURLException {
        String search_string ="LambdaTest Blog";
        String exp_title = "LambdaTest | A Cross Browser Testing Blog";

        setupThread("test_BingSearch", "test_BingSearch", "MacOS Catalina",
                "Safari", "13.0");
        WebDriver webdriver = getDriver();
        webdriver.navigate().to("https://www.bing.com");
        webdriver.manage().window().maximize();
        System.out.println("Started session");

        try {
            /* Enter the search term in the Google Search Box */
            WebElement search_box = webdriver.findElement(By.xpath("//input[@id='sb_form_q']"));
            search_box.sendKeys(search_string + Keys.ENTER);
            Thread.sleep(3000);

            /* Click on the first result which will open up the LambdaTest homepage */
            WebElement lt_link = webdriver.findElement(By.xpath("//a[.='LambdaTest | A Cross Browser Testing Blog']"));
            lt_link.click();
            Thread.sleep(5000);

            String curr_window_title = webdriver.getTitle();
            Assert.assertEquals(curr_window_title, exp_title);
            status = "passed";
        } catch (Exception e) {
            System.out.println(e.getMessage());
        }
        if (webdriver != null)
        {
            ((JavascriptExecutor) webdriver).executeScript("lambda-status="+status+"");
            webdriver.quit();
        }
    }

    @Override
    public void onExecutionFinish()
    {
        System.out.println("onExecutionFinish");
    }
}

(Implementation of Test Scenarios in ‘Test Group 2’)

package org.testnggroup; 
import org.openqa.selenium.*;
import org.testng.IExecutionListener;
import org.testng.annotations.Test;
import java.net.MalformedURLException;

public class TestNG_ToDoGroup extends Helper implements IExecutionListener {
    /* WebDriver driver; */
    /* private ThreadLocal<RemoteWebDriver> driver = new ThreadLocal<>(); */
    public static String status = "passed";

    @Override
    public void onExecutionStart()
    {
        System.out.println("onExecutionStart");
    }

    @Test (priority = 1, groups = { "ToDo" })
    public void test_Selenium4_ToDoApp_Test1() throws InterruptedException, MalformedURLException
    {
        setupThread("test_Selenium4_ToDoApp_Test1", "test_Selenium4_ToDoApp_Test1",
                "Windows 10", "Firefox", "68.0");
        WebDriver webdriver = getDriver();
        webdriver.navigate().to("https://lambdatest.github.io/sample-todo-app/");
        webdriver.manage().window().maximize();
        System.out.println("Started session");
        Thread.sleep(5000);

        try
        {
            /* Let's mark done first two items in the list. */
            webdriver.findElement(By.name("li1")).click();
            webdriver.findElement(By.name("li2")).click();

            /* Let's add an item in the list. */
            webdriver.findElement(By.id("sampletodotext")).sendKeys("Happy Testing at LambdaTest");
            webdriver.findElement(By.id("addbutton")).click();

            /* Let's check that the item we added is added in the list. */
            String enteredText = webdriver.findElement(By.xpath("/html/body/div/div/div/ul/li[6]/span")).getText();
            if (enteredText.equals("Happy Testing at LambdaTest")) {
                System.out.println("Demonstration is complete");
                status = "passed";
            }
        }
        catch (Exception e)
        {
            System.out.println(e.getMessage());
        }
        if (webdriver != null)
        {
            ((JavascriptExecutor) webdriver).executeScript("lambda-status="+status+"");
            webdriver.quit();
        }
    }

    @Test (priority = 2, groups = { "ToDo" })
    public void test_Selenium4_ToDoApp_Test2() throws InterruptedException, MalformedURLException {
        setupThread("test_Selenium4_ToDoApp_Test2", "test_Selenium4_ToDoApp_Test2",
                "MacOS Catalina", "MicrosoftEdge", "87.0");
        WebDriver webdriver = getDriver();
        webdriver.navigate().to("https://lambdatest.github.io/sample-todo-app/");
        webdriver.manage().window().maximize();
        System.out.println("Started session");
        Thread.sleep(5000);

        try
        {
            /* Let's mark done first three items in the list. */
            webdriver.findElement(By.name("li1")).click();
            webdriver.findElement(By.name("li2")).click();
            webdriver.findElement(By.name("li3")).click();
        }
        catch (Exception e)
        {
            System.out.println(e.getMessage());
        }
        status = "passed";
        if (webdriver != null)
        {
            ((JavascriptExecutor) webdriver).executeScript("lambda-status="+status+"");
            webdriver.quit();
        }
    }

    @Override
    public void onExecutionFinish()
    {
        System.out.println("onExecutionFinish");
    }
}

(Implementation of Helper Functions for creating an instance of RemoteWebDriver)

package org.testnggroup;

import java.net.MalformedURLException;
import java.net.URL;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;

public class Helper {
    protected static ThreadLocal<RemoteWebDriver> driver = new ThreadLocal<>();
    public static String username = "user-name";
    public static String access_key = "access-key";

    public void setupThread (String build, String name, String platformName,
                       String browserName, String browserVersion) throws MalformedURLException
    {
        DesiredCapabilities capabilities = new DesiredCapabilities();

        capabilities.setCapability("build", build);
        capabilities.setCapability("name", name);
        capabilities.setCapability("platformName", platformName);
        capabilities.setCapability("browserName", browserName);
        capabilities.setCapability("browserVersion",browserVersion);
        capabilities.setCapability("tunnel",false);
        capabilities.setCapability("network",true);
        capabilities.setCapability("console",true);
        capabilities.setCapability("visual",true);

        driver.set(new RemoteWebDriver(new URL("http://" + username + ":" + access_key + "@hub.lambdatest.com/wd/hub"), capabilities));
    }

    public WebDriver getDriver()
    {
         return driver.get();
    }

    public void tearDownThread()
    {
        getDriver().quit();
    }
}

Note- TestNGException is base class of all testng Exceptions i.e every exception of testng extends this.

Here are the three methods implemented in Helper.java:

1. setupThread: This method takes five arguments (Build, Name, platformName, browserName, and browserVersion), which are inline with the requirements of the LambdaTest capabilities generator. The cross browser tests would be executed on Selenium 4 Grid.

It instantiates a new ThreadLocal for each test class since the tests (in the two-class files) are executed in Parallel at the ‘Method’ level.

1. getDriver: This method returns the WebDriver object as a ThreadLocal for the parallel tests.
2. tearDownThread: This method releases the resources held by the current WebDriver object.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.testnggroup</groupId>
    <artifactId>TestNGGroups</artifactId>
    <version>1.0-SNAPSHOT</version>
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <dependencies>
        <!-- https://mvnrepository.com/artifact/com.github.lambdatest/lambdatest-tunnel-binary -->
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>6.9.10</version>
            <scope>test</scope>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-java -->
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>4.0.0-alpha-7</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-chrome-driver -->
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-chrome-driver</artifactId>
            <version>4.0.0-alpha-7</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-remote-driver -->
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-remote-driver</artifactId>
            <version>4.0.0-alpha-7</version>
        </dependency>
        <dependency>
            <groupId>io.github.bonigarcia</groupId>
            <artifactId>webdrivermanager</artifactId>
            <version>4.1.0</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-nop -->
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-nop</artifactId>
            <version>1.7.28</version>
            <scope>test</scope>
        </dependency>
        <!-- https://mvnrepository.com/artifact/junit/junit -->
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.13</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>com.github.lambdatest</groupId>
            <artifactId>lambdatest-tunnel-binary</artifactId>
            <version>1.0.4</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/com.typesafe.play/play-mailer -->
        <dependency>
            <groupId>com.typesafe.play</groupId>
            <artifactId>play-mailer_2.13</artifactId>
            <version>8.0.1</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/jakarta.mail/jakarta.mail-api -->
        <dependency>
            <groupId>jakarta.mail</groupId>
            <artifactId>jakarta.mail-api</artifactId>
            <version>2.0.0</version>
        </dependency>
        <dependency>
            <groupId>com.sun.activation</groupId>
            <artifactId>javax.activation</artifactId>
            <version>1.2.0</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <defaultGoal>install</defaultGoal>
        <plugins>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.0</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.12.4</version>
                <configuration>
                    <suiteXmlFiles>
                        <!-- TestNG suite XML files -->
                        <suiteXmlFile>testng.xml</suiteXmlFile>
                    </suiteXmlFiles>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-report-plugin</artifactId>
                <version>3.0.0-M5</version>
            </plugin>
        </plugins>
    </build>
</project>

How To Run Test Cases Within The Same Group?

As part of TestNG group examples, we explore the most basic use of groups in TestNG. In a project, you could have multiple Groups, but you might be interested in running tests that belong to a particular TestNG Group. In such cases, you can group the tests to selectively run ‘all the tests’ within a specific group.

In our example, we have two TestNG groups – Search and ToDo. Let’s look at how to run test cases within the same group (e.g., Search) in Parallel. Parallel testing in Selenium is preferred so that tests can be executed faster, and you can also make the most of the capabilities offered by the cloud-based Selenium Grid.

In testng.xml, we set the thread-count attribute to 2 and the parallel attribute to “methods.” Since we want to run the test cases implemented under the ‘Search’ group, the group is included in the <run> tag under <groups> .

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<!-- Demo 1: Test Groups (How To Create Groups?) -->
<suite name="TestNG Group Test" thread-count="4" parallel="methods">
    <test verbose="2" preserve-order="true" name="C:/Users/Lenovo/IdeaProjects/TestNGGroups">
        <groups>
            <run>
                <include name = "Search">
                </include>
            </run>
        </groups>
        <classes>
            <class name="org.testnggroup.TestNG_SearchGroup">
            </class>
        </classes>
    </test>
</suite>

Note- TestException happens when testng is unable to run test method due to any reason.

There are two test methods under the TestNG Group “Search”:

• test_GoogleSearch – Priority is set to ‘1.’
• test_BingSearch – Priority is set to ‘2.’

The specified group tests will run in Parallel as per the priority specified in the @test annotation.

 [..]
    @Test (priority = 1, groups = { "Search" })
    public void test_GoogleSearch() throws InterruptedException, MalformedURLException
    {
        String search_string =" LambdaTest";
        String exp_title = "Most Powerful Cross Browser Testing Tool Online | LambdaTest";

        setupThread("test_GoogleSearch", "test_GoogleSearch",
                "Windows 10", "Chrome", "latest");
        ..............................
        ..............................
        ..............................
    }

    @Test (priority = 2, groups = { "Search" })
    public void test_BingSearch() throws InterruptedException, MalformedURLException 
    {
        String search_string ="LambdaTest Blog";
        String exp_title = "LambdaTest | A Cross Browser Testing Blog";

        setupThread("test_BingSearch", "test_BingSearch", "MacOS Catalina",
                "Safari", "13.0");
        ..............................
        ..............................
        ..............................
    }
    [..]

As seen in the execution snapshot, two test methods under the TestNG tag – “Search” are running in Parallel:

Shown below are the execution snapshots from the IDE and the Automation Dashboard from LambdaTest it indicates that their tests were executed successfully:

How To Run Test Cases Within Multiple Groups?

TestNG lets you run test methods that belong to multiple groups. The group names are provided as an array in the groups attribute of the @test annotation. In our case, we have four test methods that are grouped under two TestNG groups. Since the test methods are a part of two different classes, the two Groups are included under two separate <test> tags.

The test methods of two different classes are run under different threads. Since we have set the parallel attribute to ‘methods,’ the methods (as per the set priority) in the respective classes are executed in parallel.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<!-- Demo 2: How to run test cases within multiple Groups -->
<suite name="TestNG Group Test" thread-count="4" parallel="methods">
    <test verbose="2" preserve-order="true" name="C:/Users/Lenovo/IdeaProjects/TestNGGroups">
        <groups>
            <run>
                <include name = "Search">
                </include>
            </run>
        </groups>
        <classes>
            <class name="org.testnggroup.TestNG_SearchGroup">
            </class>
        </classes>
    </test>

    <test name="ToDoApp">
        <groups>
            <run>
                <include name = "ToDo">
                </include>
            </run>
        </groups>
        <classes>
            <class name="org.testnggroup.TestNG_ToDoGroup">
            </class>
        </classes>
    </test>
</suite>

At first, the test methods under the group “Search” are executed in parallel. Post the successful execution of those methods, the test methods under the TestNG group “ToDo” are run in parallel.

Here are the execution screenshots that indicate that all four test methods under the groups “Search” and “ToDo” were executed successfully:

Groups Within Groups In TestNG

In this section of the TestNG tutorial, we look at how to use ‘MetaGroups’ in TestNG. Groups can include other groups. The combined group is called ‘MetaGroup.’ Groups within groups in TestNG can be helpful in cases where a ‘collection’ of test scenarios from different groups have to be run as a part of the test case.

For demonstrating how to group test cases in TestNG when Groups are within Groups, we create four test groups – Search1 and Search2 (for including the test methods that demonstrate the search functionality) & ToDo1 and ToDo2 (for including the test methods that demonstrate the ToDo App functionality).

Shown below are the code modifications for creating four TestNg groups:

public class TestNG_SearchGroup extends Helper implements IExecutionListener
{
    [..]
   @Test (priority = 1, groups = {"Search1"})
    public void test_GoogleSearch() throws InterruptedException, MalformedURLException
    {
        String search_string =" LambdaTest";
        String exp_title = "Most Powerful Cross Browser Testing Tool Online | LambdaTest";

        setupThread("test_GoogleSearch", "test_GoogleSearch",
                "Windows 10", "Chrome", "latest");
        ...................................................
        ...................................................
        ...................................................
    }

   @Test (priority = 2, groups = {"Search2"})
    public void test_BingSearch() throws InterruptedException, MalformedURLException {
        String search_string ="LambdaTest Blog";
        String exp_title = "LambdaTest | A Cross Browser Testing Blog";

        setupThread("test_BingSearch", "test_BingSearch", "MacOS Catalina",
                "Safari", "13.0");
        ...................................................
        ...................................................
        ...................................................
    }
    [..]
}

public class TestNG_ToDoGroup extends Helper implements IExecutionListener 
{
    [..]
   @Test (priority = 1, groups = {"ToDo1"})
    public void test_Selenium4_ToDoApp_Test1() throws InterruptedException, MalformedURLException
    {
        setupThread("test_Selenium4_ToDoApp_Test1", "test_Selenium4_ToDoApp_Test1",      "Windows 10", "Firefox", "68.0");
        ...................................
        ...................................
        ...................................
    }

    @Test (priority = 2, groups = {"ToDo2"})
    public void test_Selenium4_ToDoApp_Test2() throws InterruptedException, MalformedURLException
    {
        setupThread("test_Selenium4_ToDoApp_Test2", "test_Selenium4_ToDoApp_Test2",     "MacOS Catalina", "MicrosoftEdge", "87.0");
        ...................................
        ...................................
        ...................................
    }
    [..]

}

A new group named “Group1” includes the test methods implemented under the “Search1” and “Search2” groups. Another group, “Group2,” includes the test methods implemented under the “ToDo1” and “ToDo2” groups. The groups Group1 and Group2 are included as a part of a new group named ‘SuperGroup.’

The four tests which are a part of ‘SuperGroup’ are run by including the SuperGroup under the < run > tag.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<!-- Demo 3: Groups within Groups in TestNG -->
<suite name="TestNG Group Test" thread-count="4" parallel="methods">
    <test verbose="2" preserve-order="true" name="C:/Users/Lenovo/IdeaProjects/TestNGGroups">
        <groups>
            <define name = "Group1">
                <include name="Search1"/>
                <include name="Search2"/>
          </define>
            <define name = "Group2">
                <include name="ToDo1"/>
                <include name="ToDo2"/>
          </define>

            <define name = "SuperGroup">
                <include name="Group1"/>
                <include name="Group2"/>
          </define>

          <run>
             <include name = "SuperGroup"/>
          </run>
        </groups>
        <classes>
            <class name="org.testnggroup.TestNG_SearchGroup"/>
            <class name="org.testnggroup.TestNG_ToDoGroup"/>
        </classes>
    </test>
</suite>

The four test methods that are a part of the nested group named SuperGroup are executed in parallel:

All four test scenarios under ‘SuperGroup’ are executed successfully:

How To Exclude/Include Test Cases With Groups?

TestNG Groups also provides provision to ignore test scenario(s)/test methods using the <exclude> tag in the <methods> section. On similar lines, the <include> tag in the <methods> section of testng.xml is used for including the test methods as a part of the test execution process.

1. TestNG_SearchGroup class: To run test methods whose name includes “.*Bing.*” and exclude test method(s) whose name contains “.*Google.*”, we use method groups in the following manner:

<class name="org.testnggroup.TestNG_SearchGroup">
    <methods>
        <include name=".*Bing.*"/>
       <exclude name=".*Google.*"/>
   </methods>
</class>

The matching method in the class is test_BingSearch().

1. TestNG_ToDoGroup class: To run test methods whose name includes “.*ToDoApp.*” and exclude test method(s) whose name contains “.*Test1.*”, we use method groups in the following manner:

<class name="org.testnggroup.TestNG_ToDoGroup">
    <methods>
        <include name=".*ToDoApp.*"/>
        <exclude name=".*Test1.*"/>
    </methods>
</class>

The matching method in the class is test_Selenium4_ToDoApp_Test2().

Here is the execution snapshot, which indicates that the two matching test methods are run successfully on the cloud-based Selenium Grid:

How to Exclude a TestNG Group?

The TestNG framework also lets you include as well as exclude them. Exclusion of Groups in TestNG is useful in cases where there is a temporary breakage in the tests (due to some recent changes), and you do not have the time to fix them.

The tests in those groups can be temporarily deactivated to be later reactivated once the issues have been fixed. Groups can also be excluded using Regular expressions.

For demonstrating exclusion of TestNG groups, we use the <include> tag in <groups> to include TestNG groups that match the regular expression “.*ToDo.*”. The <exclude> tag in <groups> is used for excluding the TestNG group with the name “Search”.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<!-- Demo 5: Exclusion of groups (Regular Expressions) -->
<suite name="TestNG Group Test" thread-count="4" parallel="methods">
<test verbose="2" preserve-order="true" name="C:/Users/Lenovo/IdeaProjects/TestNGGroups">
    <groups>
        <run>
        <include name=".*ToDo.*"/>
        <exclude name="Search"/>
        </run>
    </groups>
    <classes>
      <class name="org.testnggroup.TestNG_SearchGroup"/>
      <class name="org.testnggroup.TestNG_ToDoGroup"/>
    </classes>
</test>
</suite>

The TestNG Group named “ToDo” matches the criteria set (i.e. “.*ToDo.*”) in the <include> tag. On the other hand, the test methods implemented in TestNG_SearchGroup are excluded as they meet the <exclude> tag requirements.

As seen below, the test methods from the matching TestNG Groups were executed successfully on the LambdaTest Selenium Grid:

Regular Expressions and TestNG Groups

TestNG uses Regular Expressions (i.e., anything that matches “.*” – Dot Star) and not WildMats (i.e., “*”) for running test methods that match the specified pattern. For demonstrating the usage of regular expressions with TestNG groups, we run test methods that include the term “Search”.

testng.xml to run test cases by matching Group Names using Regular Expressions

<!-- Demo 6: Regular Expressions and TestNG Groups -->
<suite name="TestNG Group Test" thread-count="4" parallel="methods">
<test verbose="2" preserve-order="true" name="C:/Users/Lenovo/IdeaProjects/TestNGGroups">
    <groups>
        <run>
            <include name = ".*Search.*"/>
        </run>
    </groups>
    <classes>
        <class name="org.testnggroup.TestNG_SearchGroup"/>
        <class name="org.testnggroup.TestNG_ToDoGroup"/>
    </classes>
</test>
</suite>

As seen in the testng.xml, tests under the groups – “Search” will be executed. Hence, the test methods test_GoogleSearch() and test_BingSearch() are run as per the assigned priority.

As shown below, the test methods matching the corresponding TestNG Group name are run successfully:

How To Group Your Tests On LambdaTest?

LambdaTest lets you group the automation tests using custom tags. The advantage of using custom tags is that it helps ease the Search for your test cases on the LambdaTest automation dashboard.

To group tests on Lambdatest using custom tags, create a String array containing the names of the custom tags, separated by a comma.

/* In case of just 1 tag, add 1 element in the array */
String[] customTags = {"Custom Tag"};

/* In case of multiple tags, add them in the array separated by comma */
String[] customTags = {"Tag 1", "Tag 2", "Tag 3", ...};

Now add this custom tag in the Desired Capabilities instance, as shown below:

DesiredCapabilities capabilities = new DesiredCapabilities();
[..]
// To create custom tags
capabilities.setCapability("tags", customTags);

In our case, we set the capabilities in the setupThread() method, defined in the Helper class. We create a Custom Tag that consists of the combination of Platform Name and Browser Name & Browser Version used in the cross browser testing.

public void setupThread (String build, String name, String platformName,
                       String browserName, String browserVersion) throws MalformedURLException
{
    [..]
    DesiredCapabilities capabilities = new DesiredCapabilities();

    capabilities.setCapability("build", build);
    capabilities.setCapability("name", name);
    capabilities.setCapability("platformName", platformName);
    capabilities.setCapability("browserName", browserName);
    capabilities.setCapability("browserVersion",browserVersion);
    capabilities.setCapability("tunnel",false);
    capabilities.setCapability("network",true);
    capabilities.setCapability("console",true);
    capabilities.setCapability("visual",true);
    String[] customTags = {""};
    customTags[0] = platformName + " " + browserName + " " + browserVersion;
    /* Setting Custom Tags */
    capabilities.setCapability("tags", customTags);
    /* End - Setting Custom Tags */
    [..]
}

For viewing the custom tags on the automation logs, navigate to the Automation logs on the automation dashboard, and check the applied custom tags below the test names in the left panel.

You also have the flexibility to filter the tests by selecting multiple tags at once from the filter toolbar.

Here we filter the tags matching ‘MacOS’ to locate the tests run on the macOS platform:

Custom Tags should be efficiently used for grouping test cases on LambdaTest so that it becomes easy to filter ‘required’ tests from the numerous tests executed on the Grid.

Wrapping up

In this TestNG tutorial, we had a detailed look at how to group test cases in TestNG. Groups in TestNG are useful in projects where you want to run test scenarios that match certain criteria. For example, all the cross browser tests can be put under a group named “crossbrowser” so that the tests can be easily identified. Grouping of tests also helps in temporarily disabling tests that might be causing breakage in the other tests.

As evident from the TestNG group examples, you can run tests within the same group, within multiple groups, within nested groups, and more. Regular expressions with groups and test methods are convenient when you want to include/exclude test cases (or groups) that match a certain condition.