DEV Community: Arvind Choudhary

Docker Image of your Testing Project / Dockerise test execution

Arvind Choudhary — Thu, 03 Nov 2022 16:28:39 +0000

I previously wrote about docker & testng, how it can be useful? but did you ever feel that why should we keep running tests in complex manner( on local machine/on virtual machine)? Can we make it simple enough for everyone to run tests with ease?!

What if we have a docker-image of our test code which when executed will test our app. you just have to do docker run <image-name> and that's it. your scripts shall trigger inside container and on completion you should be able to get test status on console!

Few advantages:

Anyone can execute scripts & get test results.
Development/Testing knowledge is not required.
Test execution is completely isolated.

Github repo: Arvind142/TestNG-Modular-Automation-Framework

Start with creating test suite, verifying POM!

testng.xml



<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">

<suite name="Suite">

    <!-- Listeners to help with reporting -->

    <listeners>

        <!-- Listener Class declaration -->

    </listeners>

    <test name="Test">

        <packages>

            <!-- test package declaration -->

            <package name="com.core.test_package.*"></package>

        </packages>

    </test> <!-- Test -->

</suite> <!-- Suite -->

Pom.xml



<build>

    <pluginManagement>

        <plugins>

            <plugin>

                <artifactId>maven-surefire-plugin</artifactId>

                <configuration>

                    <suiteXmlFiles>

                        <suiteXmlFile>testng.xml</suiteXmlFile>

                    </suiteXmlFiles>

                </configuration>

            </plugin>

        </plugins>

    </pluginManagement>

</build>

Trigger them to Verify it's working.

mvn clean test -Dsurefire.suiteXmlFiles=/path/to/testng.xml

Create image of your code

Dockerfile



  
  
  Base image


FROM maven:3.8.5-openjdk-11-slim

  
  
  Argument with default value


ARG suiteFile=testng.xml

  
  
  Copy code from local to image


COPY ./ /app

  
  
  Specify working directory


WORKDIR /app

  
  
  Execute command at creation of image


RUN mvn clean compile

  
  
  Command to be executed at start of container


CMD mvn test -Dsurefire.suiteXmlFiles=$suiteFile -e

Build your image

docker build -t arvindc142/framework:v1 .

Run you image

docker run arvindc142/framework:v1

Run different test suite (other than testng.xml, which was metioned as default suite)

docker run -e suiteFile=testng-karate.xml arvindc142/framework:v1

Remember to publish image to image repo so that it is available for others to leverage.

docker image push arvindc142/framework:v1

What about test reports?

since tests were executed in container then reports are within container, how can i get them? Volume Mapping

Volume Mapping

Volumes are the preferred mechanism for persisting data generated by and used by Docker containers. While bind mounts are dependent on the directory structure and OS of the host machine, volumes are completely managed by Docker.

Note: In my case reports are generated in test-output folder, I’ll map test-output of my image with my machine’s folder (i.e., test-results) so that I can get test result there! (Use PowerShell on windows)

docker run -v $PWD/test-results:/app/test-output arvindc142/framework:v1

Before Snapshot:

After executing docker run with -v option:

BONUS!!!

Let’s add selenium grid with code for UI test script execution.

above setup were only for test execution it didn’t cover selenium-based UI script execution.

Note: Remote url will be http://selenium-hub:4444/wd/hub inside framework container.

Container can connect to hub container with host as selenium-hub, i’ve also exposed port 4444 to watch script execution from your machine. you can access hub with http://localhost:4444/ui

Hub Snapshot:

Docker-compose for Selenium grid and our test script execution



version: '3.7'

  
  
  compose version details


services:

  #-SELENIUM GRID SETUP FOR AUTOMATION EXECUTION-

  selenium-hub:

    image: selenium/hub:latest

    container_name: selenium-hub

    ports:

      - 4444:4444

  chrome:

    image: selenium/node-chrome:latest

    shm_size: 2gb

    depends_on:

      - selenium-hub

    environment:

      - SE_EVENT_BUS_HOST=selenium-hub

      - SE_EVENT_BUS_PUBLISH_PORT=4442

      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443

  edge:

    image: selenium/node-edge:latest

    shm_size: 2gb

    depends_on:

      - selenium-hub

    environment:

      - SE_EVENT_BUS_HOST=selenium-hub

      - SE_EVENT_BUS_PUBLISH_PORT=4442

      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443

  firefox:

    image: selenium/node-firefox:latest

    shm_size: 2gb

    depends_on:

      - selenium-hub

    environment:

      - SE_EVENT_BUS_HOST=selenium-hub

      - SE_EVENT_BUS_PUBLISH_PORT=4442

      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443

  #-SELENIUM GRID services ended-

#-Framework image-

  # service name(container name)

  modular-automation-framework:

    # which container to use for building

    image: arvindc142/framework:v1

    # to copy project into container

    volumes:

      - './test-results/:/app/test-output/'

To spin up all containers

docker-compose up -d

Get all containers

docker ps -a

Print container logs on console

you can get container-id’s with docker ps command
docker logs -f <container-id>

To stop & remove containers

docker-compose down

References

Run TestNG from commandline

Github TestNG Project

Volume Mapping

Docker-Selenium

docker hub arvindc142/framework

TestNG testing framework — Parameterization

Arvind Choudhary — Wed, 11 May 2022 05:41:31 +0000

What is Parameterization?

Whenever we define any parameters on any of our methods, it means we are parametrizing that method. e.g. assuming areaOfSquare(int side);takes one argument which is integer and returns ares of square. it also allows you to use the same method for N. no of values.

@test method of TestNG supports parameterization, which in simple terms means it allows us to pass values to our test method at runtime.

Ways to Parameterize @test Method

@DataProvider
@Parameters

Parameterization with @DataProvider annotation

Always remember to use below 2 if plan on parameterizing your @test with data provider.

@DataProvider annotation with name attribute.
@test with attribute dataProvider having same value as specified for above step.

Code:

import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

public class DataProviderDemo {
    /**
     * always use name attribute for your dataProvider(), you can have
     * N. no of dataprovider in your Test class and with the help of dataprovider attribute of @Test(refer to line 57)
     * you can specify which dataprovider method to use.
     */

    // always returns 2D Object array
    @DataProvider(name = "testData")
    public Object[][] dataProvider(){
        // 2D Object array initialized
        Object[][] objectArray = new Object[2][2];

        /**
         * 2D Array structure
         *      |   abc     |   cde     |
         *      |   efg     |   hij     |
         *
         * what @data provider will do here is that it will send data of one row at a time to test method,
         *  then next row and so on.
         *
         *  no. of rows = no. of times your @Test will get executed
         */

        // adding value in objectArray
        objectArray[0][0] = "abc";
        objectArray[0][1] = "cde";
        objectArray[1][0] = "efg";
        objectArray[1][1] = "hij";

        // return 2D Object array
        return objectArray;
    }

    /**
     * make sure to use dataProvider attribute on your @Test annotation(refer to line 57),
     *
     * Also change you method signature according to what you are passing from your dataProvider,
     * if you are not sure on what you should do then you can also keep your parameter type as String &
     * testng would typecase it for you to string.
     *
     * no. of parameters on @test method should be same as no. of Object defined on your object array(column-wise).
     * ref line 19, we have 2 columns and 2 rows. so you should have 2 parmaters(2 columns) in your method signature.
     * (ref line no. 58)
     * -------------------------OR---------------------------
     *
     * use (String... args), here dont have to specify each parameter,
     * you will get one array which is args and you can work with it with the help of index,
     * each parameter will be stored on one index
     * assuming current case your array will have 2 indexes [0 & 1]
     * where 0th index would hold first value and 1st index would hold second value.
     * (ref line no. 63)
     */
    @Test(dataProvider = "testData") // line 57
    public void testMethod(String val1, String val2){ //line 58
        System.out.println("testMethod called with ["+val1+", "+val2+"]");
    }

    @Test(dataProvider = "testData")
    public void testMethod1(String... args){ //line 63
        System.out.println("testMethod1 called with ["+args[0]+", "+args[1]+"]");
    }
}

Output:

testMethod called with [abc, cde]
testMethod called with [efg, hij]
testMethod1 called with [abc, cde]
testMethod1 called with [efg, hij]

Parameterization with @Parameters annotation

Passing value with @Parameters is bit tricky here as we will be passing values from testng.xml

TestNG.xml


<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
    <test name="Test1">
        <parameter name="value1" value="abc"></parameter>
        <parameter name="value2" value="cde"></parameter>
        <parameter name="value3" value="klm"></parameter>
        <classes>
            <class name="ParametersDemo" ></class>
        </classes>
    </test> <!-- Test -->
    <test name="Test2">
        <parameter name="value1" value="efg"></parameter>
        <parameter name="value2" value="hij"></parameter>
        <classes>
            <class name="ParametersDemo" ></class>
        </classes>
    </test> <!-- Test -->
</suite> <!-- Suite -->

Script

import org.testng.annotations.Optional;
import org.testng.annotations.Parameters;
import org.testng.annotations.Test;

public class ParametersDemo {
    /**
     *
     *  in @Parameters you should specify parameter name
     *  and same should be present in testng.xml.
     *  refer to line 5 & 13 of testng.xml
     *
     * @param val1 parameter should be specified in test.xml
     * @param val2 parameter should be specified in test.xml
     * @param val3 is Optional, if specified well it would take specified value else defualt value
     *             refer to line 15, we have use @Optional
     */
    @Test
    @Parameters({"value1","value2","value3"})
    public void testMethod(String val1,String val2,@Optional("default") String val3){
        System.out.println("testMethod called with ["+val1+","+val2+","+val3+"]");
    }
}

Output:

testMethod called with [abc,cde,klm]
testMethod called with [efg,hij,default]

As you see above for second test tag we only specified 2 parameters instead of 3 and because of @Optional usage we didn’t get any error.

if you would have not specified @Optional and you didn’t specify parameter in testng.xml you will below error.

testMethod called with [abc,cde,klm]
org.testng.TestNGException: 
Parameter 'value3' is required by @Test on method testMethod but has not been marked @Optional or defined

Useful URLs:
Javatpoint parameters

TestNG testing framework - Introduction

Arvind Choudhary — Sun, 08 May 2022 12:13:41 +0000

What is a Testing Framework?

A testing framework is a set of guidelines or rules used for creating and designing test cases. A framework is comprised of a combination of practices and tools that are designed to help QA professionals test more efficiently.

Popular available options:

JUnit
NUnit
TestNG

Why TestNG?

TestNG is a testing framework inspired by JUnit and NUnit but introduces some new functionalities that make it more powerful and easier to use, such as:

Annotations.
Run your tests in arbitrarily big thread pools with various policies available (all methods in their thread, one thread per test class, etc…).
Test that your code is multithread safe.
Flexible test configuration.
Support for data-driven testing (with @DataProvider).
Support for parameters.
Powerful execution model (no more TestSuite).
Supported by a variety of tools and plug-ins (Eclipse, IDEA, Maven, etc…).
Embeds BeanShell for further flexibility.
Default JDK functions for runtime and logging (no dependencies).
Dependent methods for application server testing. TestNG is designed to cover all categories of tests: unit, functional, end-to-end, integration, etc…

Annotations in TestNG

Assertions

Assertions in TestNG are a way to verify that the expected result and the actual result match or not.

Expected Result: expected outcome at a point in time while testing application.
Actual Result: actual outcome at a point in time while testing application.

example:

Assert.assertEquals(actualOutcome,expectedOutcome);

Types Of Assertions

Hard Assertion: Hard Asserts are those asserts that stop the test execution when an assert statement fails, and the subsequent assert statements are therefore not validated.
Soft Assertions: In soft asserts, the subsequent assertions keep on running even though one assert validation fails, i.e., the test execution does not stop.

Example:

# Hard Assert
Assert.assertEquals(actualResult,expectedResult);
# Soft Assert
SoftAssert softassert = new SoftAssert();
softassert.assertEquals(actualResult, expecetedResult);
softassert.assertAll();

Example Usage

pom.xml

<dependency>
    <groupId>org.testng</groupId>
    <artifactId>testng</artifactId>
    <version>7.5</version>
</dependency>

code

only @test annotated methods are called test should hold test scripts, whereas @BeforeMethod/@AfterMethod methods should be used for setting up prerequisites & cleaning up for each tests.

import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;

public class Runner {
    @BeforeTest
    public void beforeTest(){
        System.out.println("method to be executed before test");
    }

    @Test
    public void test(){
        System.out.println("test method");
    }

    @AfterTest
    public void afterTest(){
        System.out.println("method to be executed after test");
    }
}

output:

method to be executed before test
test method
method to be executed after test
===============================================
Default Suite
Total tests run: 1, Passes: 1, Failures: 0, Skips: 0
===============================================

Few useful URLs:

TestNG webpage
Javatpoint tutorial

Why to use Loggers?

Arvind Choudhary — Sun, 01 May 2022 13:07:19 +0000

Loggers

In Java, logging is an important feature that helps developers to trace out the errors. Java is the programming language that comes with the logging approach. It provides a Logging API that was introduced in Java 1.4 version. It provides the ability to capture the log file.

Components of Loggers

Logger — recorder.
Handlers or Appender — Where to send logs(console, file, etc.)
Formatters or Layouts — How to display logs.

Different Loggers

Log4j: Apache Log4j is an open-source Java-based logging utility.
SLF4J: It stands for Simple Logging Facade for Java (SLF4J). It is an abstraction layer for multiple logging frameworks such as Log4j, Logback, and java.util.logging.
Logback: It is an open-source project designed as a successor to Log4j version 1 before Log4j version 2 was released.
tinylog(tinylog): It is a light weighted and open-source logger.

Why Loggers?

While building your application/framework, you would often face errors which have to be debugged. So, with the help of logs, we can easily get information about what is happening within our app or framework with a record of errors and unusual circumstances. Now, it might strike your mind that, why not use the System.out.print() statement in Java. Well, the problem with these statements is that log messages will be printed only on the console. So, once you close console, automatically, all the logs will be lost. Therefore, logs will be not be stored permanently, and are displayed one by one, as it is a single-threaded environment.
In this post we would be using Logback logger library for our demo.

Logger Level Description:

Maven dependency:

<!-- https://mvnrepository.com/artifact/ch.qos.logback/logback-classic -->
<dependency>
    <groupId>ch.qos.logback</groupId>
    <artifactId>logback-classic</artifactId>
    <version>1.2.11</version>
    <scope>test</scope>
</dependency>

logback-test.xml ( convention is to keep it in resources folder)

<configuration>

    <!-- Console Appender -->
    <appender name="console-appender" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
        </encoder>
    </appender>


    <!-- File Appender -->
    <appender name="file-appender" class="ch.qos.logback.core.FileAppender">
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
        </encoder>
        <file>trace.log</file>
        <append>false</append>
    </appender>

    <root level="ALL">
        <appender-ref ref="console-appender" />
        <appender-ref ref="file-appender" />
    </root>

</configuration>

MainClass.java

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class MainClass {
    public static void main(String[] args) {
        // initializing logger
        Logger logger = LoggerFactory.getLogger(MainClass.class);
        logger.trace("trace-log");
        logger.debug("debug-log");
        logger.info("info-log");
        logger.warn("warn-log");
        logger.error("error-log");
    }
}

In our case we are using 2 Appenders:
Console Appender : To print logs onto console.
File Appender : To send logs to file.
kindly refer to logback-test.xml where we have mentioned 2 different appenders and log format.

Console print:

21:21:32.722 [main] TRACE MainClass - trace-log
21:21:32.722 [main] DEBUG MainClass - debug-log
21:21:32.722 [main] INFO  MainClass - info-log
21:21:32.727 [main] WARN  MainClass - warn-log
21:21:32.727 [main] ERROR MainClass - error-log

trace.log

21:21:32.722 [main] TRACE MainClass - trace-log
21:21:32.722 [main] DEBUG MainClass - debug-log
21:21:32.722 [main] INFO  MainClass - info-log
21:21:32.727 [main] WARN  MainClass - warn-log
21:21:32.727 [main] ERROR MainClass - error-log

Getting started with WDM(WebDriverManager)

Arvind Choudhary — Sun, 24 Apr 2022 14:49:50 +0000

Hello Readers,

Whenever we talk about Automation Testing(Browser Automation) we should have a few things in place:

Browser (Should be on the system path).
Driver Executable (Should be on the system path), Driver Executable version should be the same as Browser Version.
Automation Library(In our case Selenium).

The second part of Point 2 is tedious. Nowadays we get at least one browser update every month or two, so we have to download the driver executable which is released for that particular browser version.

As shown in fig, chrome releases multiple browsers update each month in stable/beta/developer channels.

So, How can we make our test project in such a way that there should be no manual intervention required in getting the correct driver executables for different browsers?

WDM(WebDriverManager) utility which is created by bonigarcia could help us in automatically downloading driver version based on different browsers, browser version and OS's .

WebDriverManager

WebDriverManager is an open-source Java library that carries out the management (i.e., download, setup, and maintenance) of the drivers required by Selenium WebDriver (e.g., chromedriver, geckodriver, msedgedriver, etc.) in a fully automated manner. In addition, as of version 5, WebDriverManager provides other relevant features, such as the capability to discover browsers installed in the local system, building WebDriver objects (such as ChromeDriver, FirefoxDriver, EdgeDriver, etc.), and running browsers in Docker containers seamlessly.

Maven dependency:

<dependency>
 <groupId>io.github.bonigarcia</groupId>
 <artifactId>webdrivermanager</artifactId>
 <version>5.1.1</version>
</dependency>

Pom.xml:

minimal pom.xml dependency list for browser automation with WDM and selenium.

Script (java):

I’ve used java language to use with selenium.

Examples of how to set up driver executable for different browsers.
WebDriverManager.chromedriver().setup();
WebDriverManager.edgedriver().setup();
WebDriverManager.firefoxdriver().setup();
How to specify which driver executable we want (Explicit specification, Optional)
WebDriverManager.firefoxdriver().browserVersion("100").setup();

Console print:

Useful URL's

bonigarcia's github page

WebDriverManager website

WebDriverManager repository

Selenium Grid Setup with Docker

Arvind Choudhary — Fri, 15 Apr 2022 05:53:25 +0000

Hello readers,

from past few days I've been thinking about reducing automation test execution time by creating an environment where I could perform cross-browser and/or parallel execution.

I came-across once solution where we could leverage docker to setup selenium grid on our local machine.

Perquisite(software)

Docker

Commands to pull required images

pulling hub image
docker pull selenium/hub:latest

pulling node-chrome
docker pull selenium/node-chrome:latest

pulling node-firefox
docker pull selenium/node-firefox:latest

pulling node-edge
docker pull selenium/node-edge:latest

Commands to setup grid

creating custom network
docker network create grid

running hub image
docker run -d --rm -p 4442–4444:4442–4444 --net grid --name selenium-hub selenium/hub:latest

running node-chrome image
docker run -d --rm --net grid -e SE_EVENT_BUS_HOST=selenium-hub --name node-chrome -e SE_EVENT_BUS_PUBLISH_PORT=4442 -e SE_EVENT_BUS_SUBSCRIBE_PORT=4443 selenium/node-chrome:latest

running node-edge image
docker run -d --rm --net grid -e SE_EVENT_BUS_HOST=selenium-hub --name node-edge -e SE_EVENT_BUS_PUBLISH_PORT=4442 -e SE_EVENT_BUS_SUBSCRIBE_PORT=4443 selenium/node-edge:latest

running node-firefox image
docker run -d --rm --net grid -e SE_EVENT_BUS_HOST=selenium-hub --name node-firefox -e SE_EVENT_BUS_PUBLISH_PORT=4442 -e SE_EVENT_BUS_SUBSCRIBE_PORT=4443 selenium/node-firefox:latest

Check container status

docker ps

all your containers should have Status as Up

Grid UI:

navigate to http://localhost:4444/ui/index.html on your machine to open grid UI

Increasing instances for parallel execution on same browser

creating custom network
docker network create grid

running hub image
docker run -d --rm -p 4442–4444:4442–4444 --net grid --name selenium-hub selenium/hub:latest

running node-chrome image
docker run -d --rm --net grid -e SE_EVENT_BUS_HOST=selenium-hub --name node-chrome --shm-size="2g" -e SE_NODE_MAX_SESSIONS=8 -e SE_EVENT_BUS_PUBLISH_PORT=4442 -e SE_EVENT_BUS_SUBSCRIBE_PORT=4443 selenium/node-chrome:latest

running node-edge image
docker run -d --rm --net grid -e SE_EVENT_BUS_HOST=selenium-hub --name node-edge --shm-size="2g" -e SE_NODE_MAX_SESSIONS=8 -e SE_EVENT_BUS_PUBLISH_PORT=4442 -e SE_EVENT_BUS_SUBSCRIBE_PORT=4443 selenium/node-edge:latest

running node-firefox image
docker run -d --rm --net grid -e SE_EVENT_BUS_HOST=selenium-hub --name node-firefox --shm-size="2g" -e SE_NODE_MAX_SESSIONS=8 -e SE_EVENT_BUS_PUBLISH_PORT=4442 -e SE_EVENT_BUS_SUBSCRIBE_PORT=4443 selenium/node-firefox:latest

Grid UI:

Stopping containers & removing network

Once you are done with you testing you can remove container with the help of below command, where you run docker stop command by passing container names to stop them.

stopping container
docker stop node-chrome node-edge node-firefox selenium-hub

removing docker network
docker network rm grid