DEV Community: Akansh Singhal

Is Your Test Automation Effective? Metrics That Matter..

Akansh Singhal — Sat, 09 Nov 2024 16:17:41 +0000

In the software testing world, we often hear debates about roles—SDET vs. QA, Manual vs. Automation Testing. But instead of diving into these debates, let’s discuss something critical: how to measure the effectiveness of an automation suite. How do we know we’re building value rather than automating for the sake of automation?

The Fallacy of "Automation as Magic Bullet"

A common misconception is that automation can replace manual testing altogether, reducing the time needed to test new features. But automation isn’t a magic bullet; it doesn’t eliminate the need for thoughtful, hands-on testing. Automation is an asset primarily for regression testing—saving time only if it’s done effectively.

Many of us asked questions like "Why we are taking so much time when we have automation?". These questions often arises from the belief that automation is one time effort and if it is done then no manual effort is required for testing new features. But that is not true, It can only check existing feature, for new feature to test it we need to write new automation and cover cases. It is like development you write some line of code, similarly we need to write some line of code to test it. In fact, sometimes creating robust automation can take longer than manual testing.

To ensure your automation is truly effective, focus on these key metrics:

Test Coverage

What It Means: Effective automation ensures significant coverage of critical paths and high-risk areas in the application.
How to Quantify: Track the percentage of smoke, regression & E2E test cases for every feature and classify them into Automated, Non Automatable and Non Automated. With this we can able to say how much percentage of critical and regression test cases are automated.

Bug Detection Capability

What It Means: Automation should actively contribute to detecting issues, especially regressions.
How to Quantify: Monitor how often bugs are caught by automated tests, especially during CI/CD processes. A meaningful automation suite detects regressions before code hits production. Tracking metrics like the percentage of escaped defects (bugs caught in production) that could have been detected by automation can highlight areas for improvement. It should have proper assertions at different layers like UI, API and datasources.

Segregation of test case on area / priority

What It Means: Tests should be organized by module and priority to ensure focused testing.
How to Quantify: Prioritize tests based on their importance (e.g., smoke, sanity, regression, end-to-end). Measure execution time by category and analyze whether high-priority areas receive faster feedback than lower-priority ones. Clear categorization allows quick, effective testing based on what’s changing in the application.

Speed of Execution

What It Means: Effective automation runs tests quickly and scales well with added tests.
How to Quantify: Track the time it takes to run the entire suite and look for opportunities to optimize (e.g., parallelization, selective execution). Execution time should ideally stay within a CI/CD-compatible timeframe, allowing quick feedback without bottlenecks.

Reporting

What It Means: Automation should produce clear, actionable reports.
How to Quantify: Effective reporting includes metrics on test pass/fail rates, trends over time, and drill-down capabilities for failed cases. Good reporting also captures historical trends to aid in proactive decision-making, like identifying flaky tests or frequent failure points.

Maintenance Effort

The effort required to maintain the suite is a direct indicator of its effectiveness. High maintenance might mean tests are too brittle or aren’t built with flexibility in mind.

Automation can be transformative when done right. However, it's crucial to measure and refine it continuously to ensure it’s delivering true value. An effective automation suite frees up time for testing new features and exploratory testing, allowing your team to focus on innovation rather than constantly patching a brittle system.

By quantifying these aspects, you can ensure your automation effort is purposeful, measurable, and valuable—not just “there” for the sake of having automation.

Visual Regression Testing - Demo using TestPlane

Akansh Singhal — Thu, 15 Aug 2024 08:43:47 +0000

What is Visual Regression Testing?

Visual regression testing is like your superpower to catch those sneaky, unintended visual changes in your UI before they reach production. It’s the difference between saying, “I hope this doesn’t break anything” and knowing it won’t.

How It Works:

Visual regression tools are like high-tech spies for your UI. They take snapshots of your application before and after changes, comparing them to uncover any sneaky differences. Think of it as a high-stakes game of "Spot the Difference" with your app’s layout and design. Here’s the process in a nutshell:

Snapshot Time: Capture your UI’s look before changes (baseline snapshot).
Change Detection: Snap your UI again after changes (current release snapshot).
Comparison: The tool compares these snapshots, identifying any discrepancies.
Feedback Loop: If differences are intended, update the baseline image. If not, you’ll be alerted to potential issues.

Remember those “Spot the Difference” games from newspapers? Visual regression testing is like that, but supercharged. Imagine finding all those differences in seconds, effortlessly!

Ready to put your observation skills to the test? Let’s play a game!

Take a look at these two images:

They might look identical, but can you spot all the differences? Try your hand at it, and see how you compare to a visual regression testing tool. For a fun challenge, play the same game here.

See how it went?

I have marked all the issues which were reported by the tool!! Check out how the tool's findings matched up:

Result from tool :) All 14 issues matched!! Seems efficient.

Comments how many issues you find in the game..

The visual regression testing tool caught all 14 issues, proving its efficiency and reliability. Testing isn't just about catching bugs; it's a skill in observation and reporting. And now you’ve seen it in action!

As some wise man said,

The only way to go fast is to go well.

So this testing can help us in moving shift left and reduce our regression time.

Want to know Tool?

Now that you understand the concept of visual testing, let’s dive into how I use this tool.

A few years back, I explored a tool called Gemini by Yandex for visual regression testing. Although Gemini is deprecated now, and they created another open source tool TestPlane which we are discussing further and can be explored with the link given.

Let's see how we can install this tool on our local system

npm init testplane@latest demoProject

After installation you will get new folder called demoProject and in that folder you will see node_modules where all node packages installed will come and config file with name .testplane.conf.ts. I have added some additional configs in it (Example: Running suite on mobile and desktop chrome browsers).

export default {
    gridUrl: "http://localhost:4444/wd/hub",
    baseUrl: "http://localhost",
    pageLoadTimeout: 0,
    httpTimeout: 60000,
    testTimeout: 90000,
    resetCursor: false,
    sets: {
        desktop: {
            files: [
                "testplane-tests/**/*.testplane.(t|j)s"
            ],
            browsers: [
                "chrome-phone",
                "chrome"
            ]
        }
    },
    browsers: {
        "chrome-phone": {
            automationProtocol: "devtools",
            headless: true,
            desiredCapabilities: {
                browserName: "chrome",
                "goog:chromeOptions": {
                    args: ["--hide-scrollbars"],
                    mobileEmulation: {
                        deviceMetrics: {
                            width: 360,
                            height: 640,
                            pixelRatio: 3.0
                        },
                        userAgent: "Mozilla/5.0 (Linux; Android 4.2.1; en-us; Nexus 5 Build/JOP40D) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.166 Mobile Safari/535.19"
                    }
                }
            },
            sessionEnvFlags: {
                isMobile: true
            }
        },
        "chrome": {
            automationProtocol: "devtools",
            headless: true,
            desiredCapabilities: {
                browserName: "chrome",
                "goog:chromeOptions": {
                    args: ["--hide-scrollbars", "--start-maximized","--window-size=1920,1080"]
                }
            },
            windowSize: {
                width: 1920,
                height: 1080
            },
            sessionEnvFlags: {
                isMobile: false
            }
        }
    },
    plugins: {
        "html-reporter/testplane": {
            // https://github.com/gemini-testing/html-reporter
            enabled: true,
            path: "testplane-report",
            defaultView: "all",
            diffMode: "3-up-scaled"
        }
    }
};

We are defining desired capabilities in json format where we are defining browser properties as we defined in ChromeDriver initialisation in Selenium. You can read more about it here.

And here in example.testplane.ts we are asserting github testplane repository home page.


describe("Github Public Repository", () => {
    it("Home Page", async ({ browser }) => {
        await browser.url("https://github.com/gemini-testing/testplane");
        await browser.$("body").assertView("Full View Port",
            {
                allowViewportOverflow: true,
                captureElementFromTop: true,
                compositeImage: true,
                screenshotDelay: 1000
            }
        )
    });
});

If you see it is finding difference on last released items [18 hours] since our baseline image has [17 hours]. So it is giving failure on dynamic element which is blocker for using above code.

Since we have error on dynamic element it is observing correct but seems false positive as a tester we have information about what is dynamic or what is static.

You can define dynamic elements in ignoreElements

describe("Github Public Repository", () => {
    it("Home Page", async ({ browser }) => {
        await browser.url("https://github.com/gemini-testing/testplane");
        await browser.$("body").assertView("Full View Port",
            {
                allowViewportOverflow: true,
                captureElementFromTop: true,
                compositeImage: true,
                screenshotDelay: 1000,
                ignoreElements: [
                    "div[data-testid='latest-commit-details']",
                    "a[class='Link--primary d-flex no-underline']",
                    "div[class='react-directory-commit-age']"
                ]
            }
        )
    });
});

We can add fields like ignoreDiffPixelCount, tolerance, antialiasingTolerance, screenshotDelay if you have some specific requirements. If you see dynamic element is coming as black coloured which we define in ignoreElements. Commit and Release elements are coming black. With this we can ignore dynamic elements.

Now if we have some intended dev changes in the build and you want to change baseline image. You can go to accept mode and convert into baseline image, and then rerun the tests.

We can improve above code by defining types and constants in typescript for make it more reusable.

For executing test case use below command:

npx testplane gui

And that will open url mentioned in baseUrl defined in .testplane.conf.ts.

If you want to check baseline images testplane/screens/.
Above executable code can be found in repository

Moreover we can use this to visually test different elements as in above code we are testing whole body. In other case we can check any specific section then i.e also possible similarly.

If you have use cases related to Before test, Before suite or clean up at the end. You can check this resource.

Future Integration

Integrate TestPlane with your functional testing framework to check visual changes across:

Different Browser Versions
Different Browser Types
Different Devices

Visual regression testing is not just about identifying bugs; it's about ensuring a seamless user experience.

Conclusion

Visual regression testing is a game-changer for maintaining the visual integrity of your UI. By integrating this testing technique into your workflow, you can catch unintended visual changes early, ensuring a seamless user experience and boosting confidence in your releases.

Key Takeaways:

Spot the Differences: Visual regression testing tools act as vigilant eyes, identifying discrepancies between snapshots of your UI and alerting you to potential issues.
Supercharge Your Testing: Just like those “Spot the Difference” games we loved as kids, visual regression testing allows you to find discrepancies quickly and efficiently.
Streamline Your Workflow: Tools like TestPlane provide robust solutions for visual regression testing, integrating smoothly with your existing setup and offering powerful configuration options.
Enhance Your Coverage: Beyond simple UI checks, visual regression testing can be integrated into various scenarios, including different browsers, devices, and screen sizes, ensuring comprehensive coverage.

I have used this tool in one of my project and I found it very efficient and it reduce lots of time and small issues which can be missed.

Got any questions or experiences with visual regression testing tools? Share them in the comments below!

Journey to Integrate SonarQube Analysis on every pull request - Part 3

Akansh Singhal — Sat, 03 Aug 2024 12:42:35 +0000

Until now we have gone through two parts of this blog series:

In this part, we will explore how to efficiently report issues found on pull requests. I suggest reading the previous two parts for a better understanding of our approach.

We will divide this into two sections:
SonarQube Analysis
Post SonarQube Analysis: Updating the Report on the PR

SonarQube performs analysis on a commit level, but pull requests often contain multiple commits. This can lead to false positives if we communicate issues based solely on commit IDs.

So we are creating a storage in Dynamo DB. You can see how to set up DynamoDB on local. We are using below query to create table in dynamo db. You can refer this on github also.

aws dynamodb create-table \
    --table-name SonarIssues \
    --attribute-definitions \
        AttributeName=pull_request_id,AttributeType=N\
        AttributeName=analysed_at,AttributeType=S\
    --key-schema \
        AttributeName=pull_request_id,KeyType=HASH\
        AttributeName=analysed_at,KeyType=RANGE\
    --provisioned-throughput \
        ReadCapacityUnits=10,WriteCapacityUnits=5 --endpoint=http://localhost:8000

We are storing data in below schema in DynamoDB

@Getter
@Setter
@JsonIgnoreProperties(ignoreUnknown = true)
@JsonNaming(PropertyNamingStrategies.SnakeCaseStrategy.class)
public class AnalysisRequest {
    private long pullRequestId;
    private String repositoryName;
    private SonarQubeIssuesResponse issues;
    private String analysedAt;
}

Since we have storage of All analysis on particular PR, we will collate all the issues and if Issue count is 0 we will APPROVE the PR otherwise we will do CHANGES_REQUESTED which is similar to the way common user perform in real life.
Example State Transitions:

We have created pipeline in Jenkins on the view of above two stages:

#!groovy

pipeline {
    agent any
    parameters {
            string(name: 'REPO_OWNER', defaultValue: 'Akansh09', description: 'Git Repo Owner?')
            string(name: 'REPO_NAME', defaultValue: 'sonar-analysis', description: 'Git Repo Name?')
            string(name: 'SONAR_PROJECT', defaultValue: 'sonar-analysis', description: 'Sonar Project?')
            string(name: 'TARGET_BRANCH', defaultValue: 'main', description: 'Target branch?')
            string(name: 'SONAR_KEY', defaultValue: 'sonar-analysis', description: 'Sonar Key?')
    }

    triggers {
        pollSCM('*/5 * * * *')
    }

    stages {
     stage('SonarQube Analysis') {
       steps {
           script {
              def gitCommitHash = sh(script: 'git rev-parse HEAD', returnStdout: true).trim()
              sh "$MAVEN_HOME/bin/mvn clean verify sonar:sonar -Dsonar.projectKey=${params.SONAR_KEY} -Dsonar.projectName=${params.SONAR_PROJECT} -Dsonar.host.url=$SONARQUBE_URL -Dsonar.token=$SONARQUBE_LOGIN -Dsonar.projectVersion=$gitCommitHash"
           }
        }
      }
      stage('Post SonarQube Analysis') {
             steps {
             script {
                 sleep(time:120,unit:"SECONDS")
                     sh "$MAVEN_HOME/bin/mvn clean compile exec:java -Dexec.mainClass=\"com.sonar.analysis.SonarAnalysis\" -DSONAR_KEY=${params.SONAR_KEY} -DTARGET_BRANCH=${params.TARGET_BRANCH} -DGIT_REPO_OWNER=${params.REPO_OWNER} -DGIT_REPO_NAME=${params.REPO_NAME}"
                 }
             }
      }
   }
}

First stage we had already discussed in blog. In second stage we are doing following:

Get Corresponding Pull Request Id Source --> Target
Persist issue in DynamoDB for the current analysis
Get All Those unresolved issues which are present in the SonarQube for this PR
Approve in case 0 issues left, CHANGES_REQUESTED in case issues are still present

Since SonarQube do not have mapping on the basis of commit id or PR id. We are getting all OPEN issues and on the basis of that we are finding key which are still unclosed

public List<SonarQubeIssuesResponse.Issues> getListOfUnResolvedIssues(long pullRequestId){
        DynamoDBUtil dynamoDBUtil = new DynamoDBUtil();
        List<AnalysisRequest> allIssuesPertainingToPullRequest = dynamoDBUtil.getAnalysisRequest(DYNAMO_SONAR_TABLE, pullRequestId);

        Set<SonarQubeIssuesResponse.Issues> analysisRequestSet = allIssuesPertainingToPullRequest.stream().flatMap(analysisRequest -> {
            if(analysisRequest!=null && analysisRequest.getIssues()!=null){
                return Arrays.stream(analysisRequest.getIssues().getIssues());
            } else {
                return Stream.empty();
            }
        }).collect(Collectors.toSet());

        SonarClient sonarClient = new SonarClient();
        SonarQubeIssuesResponse sonarQubeIssuesResponse = sonarClient.getAllIssuesFromSonarQube();
        List<String> allKeys = Arrays.stream(sonarQubeIssuesResponse.getIssues()).map(SonarQubeIssuesResponse.Issues::getKey).toList();

        return analysisRequestSet.stream().filter(s->allKeys.stream().anyMatch(s1->s1.equals(s.getKey()))).toList();
    }

Sample PR where this job executed can be seen here

You can refer code with the current implementation on my github repository

So we have successfully integrated SonarQube <> Jenkins <> Github. And we are able to achieve our initial goal.

Hope this resource help you. Thank you for reading this. If you have any questions or need further information, feel free to contact me at akanshsinghal7@gmail.com.

Journey to Integrate SonarQube Analysis on every pull request - Part 2

Akansh Singhal — Fri, 02 Aug 2024 02:30:02 +0000

In this we will continue with implementing SonarQube with github pull Request. If you want to learn more about SonarQube and its integration with Github, please refer to my previous blog.

This solution involves integrating Jenkins, SonarQube, and GitHub. Let's divide this problem into two parts:

Triggering SonarQube analysis from Jenkins to GitHub as soon as a PR is raised.

Reporting issues found on the GitHub PR.
We will start by addressing the first part.

We are trying to solve animated part 1st.

You can start Jenkins on your local using resource.

Once Jenkins is ready we have to create Jenkins job of MultiBranch Pipeline

Now configure your pipeline as per below image:

Now after setting Jenkins job and adding below pipeline in code base, we are able to execute sonar analysis on this code base.

#!groovy
pipeline {
    agent any
    parameters {
            string(name: 'REPO_OWNER', defaultValue: 'Akansh09', description: 'Git Repo Owner?')
            string(name: 'REPO_NAME', defaultValue: 'sonar-analysis', description: 'Git Repo Name?')
            string(name: 'SONAR_PROJECT', defaultValue: 'sonar-analysis', description: 'Sonar Project?')
            string(name: 'TARGET_BRANCH', defaultValue: 'develop', description: 'Target branch?')
    }

    triggers {
        pollSCM('*/5 * * * *')
    }

    stages {
      stage('SonarQube Analysis') {
       steps {
           def gitCommitHash = sh(script: 'git rev-parse HEAD', returnStdout: true).trim()
           sh "$MAVEN_HOME/bin/mvn clean verify sonar:sonar -Dsonar.projectKey=Akansh09_sonar-analysis_15fb42fe-8cb3-459f-86ea-7eb5b2e2db21 -Dsonar.projectName=${params.SONAR_PROJECT} -Dsonar.host.url=$SONARQUBE_URL -Dsonar.token=$SONARQUBE_LOGIN -Dsonar.projectVersion=$gitCommitHash"
        }
      }
   }
}

Include MAVEN_HOME, SONARQUBE_LOGIN & SONARQUBE_URL in environment variable of your jenkins node.

Now part second of this solution is to have this issues persisted on the Github PR which we solve in next part of this blog.

Now we have to fetch issues from Sonarqube and comment on Github. We have to use APIs for it

curl --location 'http://127.0.0.1:9000/api/issues/search?componentKeys=${SONAR_PROJECT_KEY}&sinceLeakPeriod=true' \
--header 'Authorization: Basic ${SONAR_BASIC_TOKEN}'

This will give you all new issues come in new code changes.

curl --location 'https://api.github.com/repos/${GIT_REPO_OWNER}/${GIT_REPO_NAME}/pulls/${PR_ID}/reviews' \
--header 'Authorization: Bearer ${GIT_TOKEN}' \
--header 'Content-Type: application/json' \
--data '{
    "body": "ddd",
    "event": "REQUEST_CHANGES"
}'

The above API will comment on the pull request. Now that we know the APIs to perform both steps, there is still one more challenge: the SonarQube API does not provide context about which issues are associated with specific commit IDs. Therefore, there is no direct mapping between Commit ID <> Issue or PR <> Issue.

In Part 3 of this series, we will stitch these APIs together and create a complete solution by writing a wrapper over the SonarQube API.

If you have any questions or need further information, feel free to contact me at akanshsinghal7@gmail.com.

Journey to Integrate SonarQube Analysis on every pull request - Part 1

Akansh Singhal — Fri, 02 Aug 2024 02:26:58 +0000

What is SonarQube?

SonarQube is an open-source platform for inspecting code quality. It performs static code analysis and reports vulnerabilities, code smells, and duplications in the code under analysis.

Problem Statement:

With the increase in code velocity and the shift-left approach, it is beneficial to execute linting at multiple phases of the development cycle:

During development: Use SonarLint.
On raising a PR: Run SonarQube on new code changes.
On merging a PR: Execute SonarQube on the develop branch.

If you want to know difference between SonarQube and SonarLint, you can refer this link.

If you are using SonarQube Community version, It only supports

A Single Branch
No PR level segregation.
Segregation between Overall Code Issues and New Code Issues (Last commit on which analysis was done)

However, if you are using the Developer Edition of SonarQube, you can utilize its built-in features to keep your code clean. This edition does come at a cost.

To address the limitations of the Community Edition, we need to implement a custom solution. This involves writing a wrapper and creating a multi-branch Jenkins pipeline.

We will use some public APIs from GitHub and SonarQube to make this fully functional. In this blog, I will share my approach to solving this problem.

Installation & Setup

You can run sonarqube using docker command

$ docker run -d --name sonarqube -e SONAR_ES_BOOTSTRAP_CHECKS_DISABLE=true -p 9000:9000 sonarqube:latest

More can be checked here

Either use any source code management tool like Github, BitBucket
Other one is to Create New Project

Using Github

If we are setting up with direct source code tool then we should be able to create communication channel between Github <> SonarQube. If Sonar is under some firewall or VPN. Please don't go ahead with this approach.

Now for this we need to perform following steps:

Create App in Github :
- Go to Settings
- Move to Developer's Setting
- Create on New Github app button

Add App Name & other mandatory details
Now Save App
You will get option to generate private key and client secret
Add redirection and callback url as http://localhost:9000/ If you do not have any valid page.
Add permission for Pull Request, Webhook, Projects for read & write access.
Save the application and install it by moving to instal menu.

Once this is done move to SonarQube and Enter github app details like AppId, Client Id, Client Secret & Private Key and Save it. Once that is done, It will ask for authorization, Give permission.
Now you will get the option to select all those projects which you want to import in sonarqube from github.
After selection you will get projects like below image:

Now our task is to select way of analysis

Going with the Github Actions only in this approach, because I want to get most of the benefit from Github. You will get screen like below on selecting Github Actions:

Now go to repository which you want to integrate and enter generated token there in Secrets field and sonar url in other secret.

Now we want to execute every pull request raised to develop, we can use below YAML file to do so.

name: Build

on:
  pull_request:
        branches: [develop]

jobs:
  build:
    name: Build and analyze
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0  # Shallow clones should be disabled for a better relevancy of analysis
      - name: Set up JDK 17
        uses: actions/setup-java@v1
        with:
          java-version: 17
      - name: Cache SonarQube packages
        uses: actions/cache@v1
        with:
          path: ~/.sonar/cache
          key: ${{ runner.os }}-sonar
          restore-keys: ${{ runner.os }}-sonar
      - name: Cache Maven packages
        uses: actions/cache@v1
        with:
          path: ~/.m2
          key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
          restore-keys: ${{ runner.os }}-m2
      - name: Build and analyze
        env:
          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
          SONAR_HOST_URL: ${{ secrets.SONAR_HOST_URL }}
        run: mvn -B verify org.sonarsource.scanner.maven:sonar-maven-plugin:sonar -Dsonar.projectKey=${{Project key generated}} -Dsonar.projectName='card'

Let's raise a PR:

After raising PR, actions will start executing itself:

If you see analysis got failed:

The initial attempt failed because the SonarQube server is not reachable from GitHub's public IP. For the integration to work, our SonarQube instance must be publicly accessible, which is often challenging in organizational settings. In my case, SonarQube is set up on my local machine (127.0.0.1:9000), which is not accessible to GitHub.

Above PR and actions can be checked on https://github.com/Akansh09/card/actions

Thank you for reading this. In the next part, we will discuss the second approach: integrating GitHub, Jenkins, and SonarQube.

If you have any questions or need further information, feel free to contact me at akanshsinghal7@gmail.com.

Automating Google Mails in your Test Suites

Akansh Singhal — Sat, 04 Feb 2023 07:37:58 +0000

Do you have use cases for validating mails or reading data from them?
Are you trying to automate Google Mail through Frontend?
Are you blocked by google while login through frontend?
Are you validating your mails on Mailinator or yopmail not on google?

Then you are on correct location. I will be explaining how easy is to validate mails through java. You can Follow below steps for verifying emails using Java:

Create App password

As a very first step create an app password for authentication purpose on Gmail. With this you need not to worry about two factor authentication.

Move to Manage your google Account Section
Select Security
Select App password
Select Mail as the app and Select Any device
Click on Generate button
Save the token for future purpose

Create Folder and filter

Now to have distinction between mails you want to validate and other random emails in your inbox.

Click on search bar
And Click on icon on right side
And Enter values according to your use case
And Click on click filter button

And Click on Apply the label
Then Click on Create Filter button as shown below

Code

Now we will add code where we will be using above two things

Include dependency in pom.xml

<!-- https://mvnrepository.com/artifact/javax.mail/javax.mail-api -->
<dependency>
    <groupId>javax.mail</groupId>
    <artifactId>javax.mail-api</artifactId>
    <version>1.5.4</version>
</dependency>

Add code to read Mail

Properties properties = new Properties();  
properties.put("mail.store.protocol", "imaps");
properties.put("mail.imap.host", "imap.gmail.com");
properties.put("mail.imap.port", "993");
properties.put("mail.imap.starttls.enable", "true");
Session session = Session.getDefaultInstance(properties);
Store store=session.getStore("imaps");
store.connect("imap.gmail.com",USER_NAME, PASSWORD);
//Put folder you have created instead of Mail Folder
Folder folder = store.getFolder("MailFolder"); 
folder.open(Folder.READ_ONLY);
int totalMessagesInFolder=folder.getMessageCount();
System.out.println(totalMessagesInFolder);

Message[] message = folder.getMessages();
//Prints Subject
System.out.println(message[0].getSubject());

//Get Email Body
String value = getTextFromMessage(message[0]);
System.out.println(value);

Now next step is to parse the Email Body since this contains multiple parts like HTML, ICS file, Attachment so it is bit complicated.

public static String getTextFromMessage(Message message) throws MessagingException, IOException {
        if (message.isMimeType("text/plain")) {
            return message.getContent().toString();
        } 
        if (message.isMimeType("multipart/*")) {
            MimeMultipart mimeMultipart = (MimeMultipart) message.getContent();
            return parseMultiPart(mimeMultipart);
        }
        return "";
    }

    public static String parseMultiPart(MimeMultipart multipart) throws IOException, MessagingException {
        String result = "";
        for (int j = 0; j < multipart.getCount(); j++) {
            BodyPart bodyPart = multipart.getBodyPart(j);
            String disposition = bodyPart.getDisposition();
            if (disposition != null && (disposition.equalsIgnoreCase("ATTACHMENT"))) { 
                System.out.println("Mail have some attachment");
                DataHandler handler = bodyPart.getDataHandler();
                System.out.println("file name : " + handler.getName());                                 
            }
            else { 
                if(bodyPart.isMimeType("text/plain")) {
                    return bodyPart.getContent().toString();
                } else {
                    result+=parseBodyPart(bodyPart);
                }
            }
        }
        return result;
    }

    public static String parseBodyPart(BodyPart bodyPart) throws MessagingException, IOException { 
        if (bodyPart.isMimeType("text/html")) {
            return "\n"+bodyPart.getContent().toString();
        } 
        if (bodyPart.getContent() instanceof MimeMultipart){
            return parseMultiPart((MimeMultipart)bodyPart.getContent());
        }
        return "";
    }

With this you can automate your email workflows like reading OTP, verifying mail after particular actions in your automation. You can also use below methods for further validations

message.getAllRecipients() //For verifying Recipient email address 
message.getFrom() //For verifying Sender's address
message.getReceivedDate() //For verifying Received date & time

Selenium: ExpectedCondition vs ExpectedConditions

Akansh Singhal — Tue, 31 Jan 2023 15:47:00 +0000

Selenium is a popular testing framework used for automating web applications. One of the key features of Selenium is the ability to wait for specific conditions to be met before performing actions on a web page. The two main methods used for waiting in Selenium are ExpectedCondition and ExpectedConditions. In this post, we'll explore the difference between these two methods and when to use them.

What are ExpectedCondition and ExpectedConditions?

ExpectedConditions is utility class which we use to apply waits in Explicit Wait in Selenium. This class includes conditions on which wait can be applied. But what we have to do, if we have to apply wait on some custom condition? Then ExpectedCondition come into picture.

ExpectedCondition is an interface which inherits (extends) Function interface which is introduced in Java8 along with Consumer, Supplier and Predicate interface.

What is Function Interface and application in ExpectedCondition?

Function Interface is a type of functional interface that receives one argument and returns a value after the required processing. It has one method named apply which takes one parameter and return a value with specified type in second argument.

public interface ExpectedCondition<T> extends Function<WebDriver, T> {}

As we can see in above code snippet, we are inheriting Function interface with one argument as WebDriver (which act as input) and T as return type which depends on value passed while implementing this interface.

Let's see how this can help us while creating automation framework. As we have to use explicit wait with some custom condition which is not provided by ExpectedConditions class. Then we can implement this interface and override apply method given by this ExpectedCondition interface (which is inherited from Function interface).
Below code snippet helping us in creating a condition to have wait till Javascript get completely loaded on page.

ExpectedCondition<Boolean> expectedConditon=new ExpectedCondition<Boolean>() {
        @Override
        public Boolean apply(WebDriver driver) {
            return ((JavascriptExecutor) driver).executeScript("return document.readyState")
                    .toString().equals("complete");
        }
};

In above code snippet we are using anonymous class (class without any name) to implement ExpectedCondition<T> interface by passing Boolean as return type. And then implementing apply method which this interface inherits from Function<F, T> interface (a sort of functional interface).

Best Practices for Using ExpectedCondition and ExpectedConditions

Here are some best practices to keep in mind when using ExpectedCondition and ExpectedConditions in your Selenium tests:

Use ExpectedConditions when possible to save time and ensure consistency in your tests.
Use ExpectedCondition when you need to define a custom condition that is not provided by Selenium.
Always specify a timeout when using ExpectedCondition or ExpectedConditions to avoid waiting indefinitely.
Use the ExpectedConditions.not() method to wait for an element to disappear from the page.
Use the ExpectedConditions.and() and ExpectedConditions.or() methods to create more complex conditions.

Conclusion

ExpectedCondition and ExpectedConditions are powerful tools in Selenium for waiting for specific conditions to be met before performing actions on a web page. By understanding the difference between these two methods and when to use them, you can write more efficient and effective Selenium tests. Remember to always use best practices like specifying a timeout and using pre-defined conditions when possible. ExpectedCondition is an interface and ExpectedConditions is a collection of static methods in Selenium that return ExpectedCondition objects.

Prospect or Usage of ChatGPT in Test Automation

Akansh Singhal — Thu, 22 Dec 2022 17:49:28 +0000

As ChatGPT shattered Internet with its launch, So I tried to explore it in terms of Test Automation

What is ChatGPT?

ChatGPT is a natural language processing (NLP) model developed by OpenAI that can generate human-like responses to a given prompt. In recent years, the use of ChatGPT has gained popularity in various industries, including test automation. In this post, we'll explore the prospect of ChatGPT in test automation and how it can be used to improve the testing process.

How to use ChatGPT in Test Automation?

Let's start with first step of software testing i.e Requirement Understanding and Test Case Creation. So I have send basic requirement of ecommerce search and asked it to write test cases for it.
Query to ChatGpt:

Feature: User is able to search on different products in ecommerce site
Inputs: Price Range, Item Category, Size, Color, Brand

Can you write all test cases for it?

And response by ChatGPT is truly amazing.. You wanna know?

It didn't stop here, you can also get Cucumber Feature file.

Feature: Search for products on ecommerce site
  As a user, I want to be able to search for products on the ecommerce site by specifying different search criteria such as price range, item category, size, color, and brand.

Scenario: Search for products by price range
  Given I am on the ecommerce site's home page
  When I enter a price range in the search bar and press enter
  Then I should see a list of products within the specified price range

Scenario: Search for products by item category
  Given I am on the ecommerce site's home page
  When I select an item category from the dropdown menu and press search
  Then I should see a list of products in the specified category

Scenario: Search for products by size
  Given I am on the ecommerce site's home page
  When I select a size from the size dropdown menu and press search
  Then I should see a list of products in the specified size

Scenario: Search for products by color
  Given I am on the ecommerce site's home page
  When I select a color from the color dropdown menu and press search
  Then I should see a list of products in the specified color

Scenario: Search for products by brand
  Given I am on the ecommerce site's home page
  When I enter a brand name in the search bar and press enter
  Then I should see a list of products from the specified brand

Reaction:

Then my expectation increase more, and I asked it to automate APIs. So below is my query to it.

Can you automate below curl request using rest assured and java?

curl --location --request POST 'https://localhost:8080/login' \
--header 'Content-Type: application/json' \
--data-raw '{
    "username":"akansh@yopmail.com",
    "password":"12222"
}'

And response can shock you:

Till now it is meeting my expectations. So as per human basic nature, you can't be satisfied with what you have, you always need more. So I pass query to find xpath:

Can you write xpath of 2nd <li> tag in below html?

"<div> <li id='akansh'>Abc</li>
<div>
<li>Abc</li>
</div>
</div>"

What do you think it meets my expectation?

No, this tool doesn't meet my expectation here, since it doesn't provide me relative xpath. Moreover, here I provide a dummy HTML block, in real life situation we have 1000s Line of code and it has word limit.

It is definitely reduce our effort but it can not completely take over us right now. Following are reason why I think so:

Maintainability & Reusability of code:
As we all work on some sort of framework (TDD, BDD etc.) but it provide us solution that uses methods which make us framework useless. Moreover there will be duplicity of code which in return increase maintenance cost.
Exploration of locator:
As we seen above, it is unable to find relative xpaths/locators, which are basic building block of UI Automation.
Database Validation:
As each organization wants to keep their Database schema private, so it will be conflict of interest if someone tries to write DB query from it or want to validate data from it.
Integration Test Cases:
You can't rely on ChatGPT to write integration test cases for you. It can't map the impact area between modules and features.
Can assume things/Biased:
As tester can never assume requirement or flows, but ChatGPT can assume things since it uses AI and which can lead to miss of potential bug. So we can't totally rely on it. It can add biases in test case formation or data generation on the basis of its AI model.

Conclusion:

ChatGPT is optimzed version of Google Search or Stack Overflow where you can put your query and you will get some instant solution instead of moving to different web links. And it also uses some AI to predict particular solution as per your requirement while current solutions are based on static webpages. ChatGPT can be used in test automation to generate test cases, test data, and test scripts. This can save time and effort in writing and maintaining tests.