DEV Community: Ish Abbi

Generating HTML reports for your tests via Gradle

Ish Abbi — Wed, 24 Jun 2020 14:52:05 +0000

In the test automation world, visualizing the results of your tests have an equal priority in conjunction with the tests being executed. How will you be able to make the most out of the tests being executed? Would a simple report having Pass/Fail ratio along with the screenshots suffice? or the reports need to be more comprehensive? maybe the report needs to keep a trend of the tests being executed over time? or needs to have a documented step for each action?

There are many frameworks for your reporting needs (two of the most notable ones are Allure and Extent reports) in the market today, even gradle provides an in-house html report on execution of a junit/testng test.

In case your project is on gradle and you want to customize the reports or your tests run via a CLI, here I try to explain an easy way to start with generating your own custom HTML reports via gradle tasks using the Groovy MarkupBuilder.

What is Gradle?

Gradle is an open-source build automation tool like Maven and Ant. It is build upon the same concepts as its predecessors with the only difference being that it is better in terms of performance and provides a Groovy-based DSL allowing you; the programmer to be more flexible. It supports languages like Java, Scala and Groovy with support for more languages planned for the future.

In case, your project is not part of the above system of supported languages you can create a project with the basic gradle settings too. Due to the DSL and task based structure of gradle, it provides you a lot of flexibility. As long as you can code the logic as a task, there are many things you can do in your project.
You can read further about gradle tasks and build lifecycle here.

The Groovy MarkupBuilder

groovy.xml.MarkupBuilder is the Groovy helper class which will help us in generating the html file for our reports.

To create a tag using the markup builder you need to add a method call with the name of the tag that you want to create. Assuming the xmlMarkup as the object initialized, the below code :

xmlMarkup.myCustomTag()

will create a tag myCustomTag in the XML/HTML format

<myCustomTag></myCustomTag>

To add text to the tag created, just add a string as an argument. So, the below code :

xmlMarkup.myCustomTag("Lorem Ipsum")

will produce :

<myCustomTag>Lorem Ipsum</myCustomTag>

To add any attribute to a tag you can add key values as arguments to the call. So the below code :

xmlMarkup.myCustomTag("class": "container", "name": "typical", "Lorem Ipsum")

will create the your tag, with the following attributes :

<myCustomTag class="container" name="typical">Lorem Ipsum</myCustomTag>

To create nested tags, you need to add groovy closures :

xmlMarkup.myCustomTag("class": "container", "name": "typical") { 
    tag1("class": "table") { 
        tag2("class": "turner","Lorem Ipsum") 
    } 
}

To create self-closing tags you can use mkp.yeild() method call.

You can read more about MarkupBuilder here.

Usage :

A typical gradle flow contains of tasks which are executed in a respective phase of the build's cycle. A task in gradle's terminology can be work such as executing tests for a build, copying a particular file to a different destination, executing a CLI command, archiving a folder, etc.
The example shown below is of a test output in the JUnit's XML format. For our use case, we'll be creating a task called htmlReport which will read the data from a XML and write it down to the report as an HTML file.

Given we have a XML file as below :

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<testsuites>
  <testsuite name="Test Suite" tests="3" failure="3" timestamp="2018-10-04T06:58:18.063Z">
    <testcase name="Test 1" classname="com.test.demo" time="92.689">
      <failure message="Step Failed: Element not found"/>
      <system-out>Some Test Related Output</system-out>
    </testcase>
    <testcase name="Test 2" classname="com.test.demo" time="95.604">
      <failure message="Step Failed: Element not found"/>
      <system-out>Some Test Related Output</system-out>
    </testcase>
    <testcase name="Test 3" classname="com.test.demo" time="93.25">
      <failure message="Step Failed: Element not found"/>
      <system-out>Some Test Related Output</system-out>
    </testcase>
  </testsuite>
</testsuites>

In order to convert the above into a custom HTML report, we'll start with the below skeletal build.gradle file.

import groovy.xml.MarkupBuilder
group '<project-name>'
version '1.0-SNAPSHOT'
apply plugin: 'base'
task htmlReport {
     // Write code here
}

The apply plugin: 'base' provides some tasks and conventions that are common to most builds. It also adds a structure to the build that promotes consistency in how they are run.

We can parse the above written xml in Groovy using XMLParser. The following code parses the xml document for us :

def xmlFile = new XmlParser().parse(filename)

You can fetch values of a tag's attributes by following the syntax :

xmlFile.<tagName>['@<attribute>']

//To get name of a testsuite from the given xml
xmlFile.testsuite['@name']

//To get the name of the first test case present inside the testsuite
xmlFile.testsuite[0].testcase[0]['@name']

Now that we know how to read from the xml file. We start by initializing the MarkupBuilder for creating our html file :

def xmlWriter = new FileWriter(file("${project.buildDir}/index.html"))
def xmlMarkup = new MarkupBuilder(xmlWriter)

As an HTML convention we'll need to add the DOCTYPE tag

xmlWriter.write("<!DOCTYPE html>\n")

Using our learnings from the The Groovy MarkupBuilder section, we can create the HTML documents like the one below :

xmlMarkup.html() {
  head() {
    meta(charset: "utf-8")
    meta(name: "viewport", content:"width=device-width, initial-scale=1, maximum-scale=1")
    title("Test Report")
    link(rel: "stylesheet", href: "https://cdnjs.cloudflare.com/ajax/libs/materialize/1.0.0-beta/css/materialize.min.css", media: "screen,projection") { mkp.yield("") }
    link(rel: "stylesheet", href: "https://fonts.googleapis.com/icon?family=Material+Icons") { mkp.yield("") }
 }
  body(){
    header(){
        div("class":"nav-wrapper"){
            nav(class: "blue darken-3") {
                a(class: "brand-logo", href: "#",id: "logo-container") {
                img(class: "logo-img",src: "") {
                    mkp.yield("")
                }
            }
            ul(class: "center flow-text", "TEST RESULTS")
            }
        }
    }
    div(class: "container") {
    div(style: "margin-bottom: 50px;") {
        table(class: "striped responsive-table", id: "example") {
            thead() {
                tr() {
                    th(class: "center","Test Suite")
                    th(class: "center","Passed")
                    th(class: "center","Failed")
                    th(class: "center","Total")
                }
            }
            tbody() {
            xmlFile.testsuite.each { testsuite ->
            tr() {
                td(class: "center",) {
                    a(href: "#",testsuite['@name'])
                }
                td(class: "center",Integer.valueOf(Integer.valueOf(testsuite['@tests']) - Integer.valueOf(testsuite['@failure'])))
                td(class: "center",testsuite['@failure'])
                td(class: "center",testsuite['@tests'])
            }
        }
    }
}
    footer(class: "page-footer blue darken-3") {
        div(class: "footer-copyright") {
            div(class: "container", "2018 - Sprinklr") {
                a(class: "grey-text text-lighten-4 right", "")
            }
        }
    }
}

So our build.gradle file should look like this :

import groovy.xml.MarkupBuilder
group '<project-name>'
version '1.0-SNAPSHOT'
apply plugin: 'base'
task htmlReport {
    doLast {
    def xmlFile = new XmlParser().parse(new File("xmlFileName"))
def xmlWriter = new FileWriter(file("${project.buildDir}/index.html"))
def xmlMarkup = new MarkupBuilder(xmlWriter)
xmlWriter.write("<!DOCTYPE html>\n")
xmlMarkup.html() {
    head() {
        meta(charset: "utf-8")
        meta(name: "viewport", content:"width=device-width, initial-scale=1, maximum-scale=1")
        title("Test Report")
        link(rel: "stylesheet", href: "https://cdnjs.cloudflare.com/ajax/libs/materialize/1.0.0-beta/css/materialize.min.css", media: "screen,projection") { mkp.yield("") }
        link(rel: "stylesheet", href: "https://fonts.googleapis.com/icon?family=Material+Icons") { mkp.yield("") }
    }
    body(){
        header(){
            div("class":"nav-wrapper"){
                nav(class: "blue darken-3") {
                    a(class: "brand-logo", href: "#",id: "logo-container") {
                        img(class: "logo-img",src: "") {
                            mkp.yield("")
                        }
                    }
                    ul(class: "center flow-text", "TEST RESULTS")
                }
            }
        }
        div(class: "container") {
            div(style: "margin-bottom: 50px;") {
                table(class: "striped responsive-table", id: "example") {
                    thead() {
                        tr() {
                            th(class: "center","Test Suite")
                            th(class: "center","Passed")
                            th(class: "center","Failed")
                            th(class: "center","Total")
                        }
                    }
                    tbody() {
                        xmlFile.testsuite.each { testsuite ->
                            tr() {
                                td(class: "center",) {
                                    a(href: "#",testsuite['@name'])
                                }
                                td(class: "center",Integer.valueOf(Integer.valueOf(testsuite['@tests']) - Integer.valueOf(testsuite['@failure'])))
                                td(class: "center",testsuite['@failure'])
                                td(class: "center",testsuite['@tests'])
                            }
                        }
                    }
                }
            }
            xmlFile.testsuite.each { testsuite ->
                h2("Suite Name : " + testsuite['@name'])
                h5("Total Tests Executed : " + testsuite['@tests'])
                h5("Total Tests Failed : " + testsuite['@failure'])
                h5("Date : " + testsuite['@timestamp'])
                div(style: "margin-bottom: 50px;") {
                    table(class: "striped highlight responsive-table") {
                        thead() {
                            tr() {
                                th(class: "center","Test Name")
                                th(class: "center","Result")
                                th(class: "center","Failure Reason")
                                th(class: "center","Test URL")
                            }
                        }
                        tbody() {
                            testsuite.testcase.each { testcase ->
                                if(testcase.failure.size() <= 0) {
                                    tr(class: "green lighten-5") {
                                        td(class: "center",testcase['@name'])
                                        td(class: "center","Passed")
                                        td("")
                                        td(class: "center") {
                                            a(href: testcase.'system-out'.text(), target: "_blank", testcase.'system-out'.text())
                                        }
                                    }
                                } else {
                                    tr(class: "red lighten-5") {
                                        td(class: "center",testcase['@name'])
                                        td(class: "center", "Failed")
                                        td(class: "center", testcase.failure['@message'].toString().replace("[","").split("More info")[0])
                                        td(class: "center") {
                                            a(href: testcase.'system-out'.text(), target: "_blank", testcase.'system-out'.text())
                                        }
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
    footer(class: "page-footer blue darken-3") {
        div(class: "footer-copyright") {
            div(class: "container", "Ish Abbi") {
                a(class: "grey-text text-lighten-4 right", "")
            }
        }
    }
    script(type: "text/javascript", src: "https://cdnjs.cloudflare.com/ajax/libs/materialize/1.0.0-beta/js/materialize.min.js") { mkp.yield("") }
}
    }
}

All you need to do now is to place this report generation task after the task which executes your tests and you should be able to view your custom reports upon success/failure of the tests.

You can also refer to the example code hosted on GitHub.

Understanding the Selenium Webdriver

Ish Abbi — Fri, 19 Jun 2020 14:01:18 +0000

Over the course of time, my pursuit for understanding the internals of Selenium compelled me to crawl the corners of the internet in search for answers. I was searching for something that can help me connect the dots between the the browser and selenium.

With countless blogs and documentation fuelling my experiments, here is what I’ve been able to learn of the WebDriver and the W3C WebDriver Protocol.

What is a WebDriver?

According to the selenium documentation :

WebDriver drives a browser natively, as a user would, either locally or on a remote machine using the Selenium server, marks a leap forward in terms of browser automation.

Selenium WebDriver refers to both the language bindings and the implementations of the individual browser controlling code. This is commonly referred to as just WebDriver.

Selenium WebDriver is a W3C Recommendation

To further simplify the understanding, I define the WebDriver as :

WebDriver is a server sitting between the test code and the browser imitating user actions.

It implements a set of REST-ish like APIs mentioned as part of the W3C WebDriver Protocol which perform the relevant actions such as clicks, sending text to input, etc as per the specifications.

These implementations are generally based on the browsers. Eg : Chrome and Firefox have a totally different implementation of the same set of standards. Hence, the requirement of GeckoDriver and ChromeDriver.

Running the WebDriver locally :

In order for the code to interact with the browser, it relevant WebDriver need to be started. As mentioned previously, this will help us in serving the actions we want to automate on the browser. In this example, we'll be using Chrome whose's WebDriver instance can be started using the below command :

./chromedriver --port 9515

This starts the instance of the WebDriver which can be used to automate browser actions. Here, the driver is listening to the port 9515, so all the request we make should be directed on the same port.

Interacting with the WebDriver :

Now, in order to start a new instance of the browser, one needs to send a POST request to the /session endpoint along with the capabilities of the browser.

curl -v -XPOST "http://localhost:9515/session" -d '{"capabilities":{"firstMatch":[{"browserName":"chrome","goog:chromeOptions":{"args":[],"excludeSwitches":["enable-automation"],"extensions":[],"prefs":{"credentials_enable_service":false,"profile.default_content_setting_values.notifications":1,"profile.default_content_settings.popups":0,"profile.password_manager_enabled":false}}}]},"desiredCapabilities":{"browserName":"chrome","goog:chromeOptions":{"args":[],"excludeSwitches":["enable-automation"],"extensions":[],"prefs":{"credentials_enable_service":false,"profile.default_content_setting_values.notifications":1,"profile.default_content_settings.popups":0,"profile.password_manager_enabled":false}}}}'

The output of which gives me the session id of the browser, which in turn is used in performing the user actions. The output of the command above gave the following response :

{
  "value": {
    "capabilities": {
      "acceptInsecureCerts": false,
      "browserName": "chrome",
      "browserVersion": "81.0.4044.138",
      "chrome": {
        "chromedriverVersion": "81.0.4044.69 (6813546031a4bc83f717a2ef7cd4ac6ec1199132-refs/branch-heads/4044@{#776})",
        "userDataDir": "/var/folders/mx/qj98dnwj50vfpk4gr8z0__jr0000gn/T/.com.google.Chrome.B4BYFs"
      },
      "goog:chromeOptions": {
        "debuggerAddress": "localhost:65302"
      },
      "networkConnectionEnabled": false,
      "pageLoadStrategy": "normal",
      "platformName": "mac os x",
      "proxy": {

      },
      "setWindowRect": true,
      "strictFileInteractability": false,
      "timeouts": {
        "implicit": 0,
        "pageLoad": 300000,
        "script": 30000
      },
      "unhandledPromptBehavior": "dismiss and notify",
      "webauthn:virtualAuthenticators": true
    },
    "sessionId": "8294d507e58e23d05cb03cfd26540530"
  }
}

With the session id with me, I can now use it to further automate the browser action like navigating to a webpage, for which we need to hit the /session/{sessionId}/url endpoint along with the url as payload :

curl -v -XPOST "http://localhost:9515/session/8294d507e58e23d05cb03cfd26540530/url" -d '{"url" : "https://www.amazon.com"}'

The response of which after successfully navigating to the url (in this case amazon) is null as per the specification if things are fine.

{"value":null}

And the same session id can be used to delete my browser session by sending the DELETE request to the /session/{sessionId} endpoint, effectively quitting my browser :

curl -XDELETE "http://localhost:9515/session/8294d507e58e23d05cb03cfd26540530"

The response of which is again null as per the specification.

{"value":null}

The above example give better insights of the working of selenium. The session that it creates and how that is used w.r.t the automation of user actions. Hope this article finds its way for the curious soul searching for answers.