DEV Community: Suparnakhamaru1

How To Set Up Continuous Integration With Git and Jenkins?

Suparnakhamaru1 — Tue, 22 Nov 2022 13:49:38 +0000

There are various CI/CD tools such as CircleCI, TeamCity, Bamboo, Jenkins, GitLab, Travis CI, GoCD, etc., that help companies streamline their development process and ensure high-quality applications. If we talk about the top CI/CD tools in the market, Jenkins is still one of the most popular, stable, and widely used open-source CI/CD tools for building and automating continuous integration, delivery, and deployment pipelines smoothly and effortlessly.

In this article, we shall explore Jenkins in detail to further understand how it’s widely used for test automation of application development tasks, running tests, and final deployment into production. We will then learn how to set up continuous integration with Git Jenkins.

This smoke testing tutorial covers what smoke testing is, its importance, benefits, and how to perform it with real-time examples.

Getting Started With Jenkins Continuous Integration

Before starting continuous integration with Jenkins and Git, let us walk through some of the most common terminologies involved in building efficient CI/CD pipelines across various SDLC projects:

Continuous Integration
Continuous Delivery
Continuous Deployment
Source Control Management (SCM)

Let us understand what happens in each of the above stages:

1. Continuous Integration

Continuous Integration
Building Software implementation in a ‘Development’ environment.
Performing Continuous Dev Testing such as:
Unit Testing
Layout Testing
Continuous Delivery
Deploying every developed build on the ‘Staging’ environment.
Performing Continuous Automated Testing such as:
Integration Testing
Visual Testing / Screenshot Testing
System Testing (End to End Testing)
Acceptance Testing
Continuous Deployment
Deploying every build on ‘Production’ environment automatically, after passing all tests in Development and Staging environments
Source Control Management (SCM)
Contains all the developed source code of the project
Maintains all the current and past state of the code/project

Source Control Management is also referred to as Version Control Software (VCS) and Revision Control System (RCS). While there are several source control management tools available in the industry, Git is the most popular and widely used one as of date, while there are others like SVN, TFS, Mercurial, CVS as well. This is another reason why continuous integration with Jenkins and Git is one of DevOps professionals’ most preferred integrations.

Before getting started with continuous integration with Jenkins and Git, let us dive a little deep into Jenkins and its setup.

Jenkins Overview

Jenkins was built by Kohsuke Kawaguchi in 2004 and is one of the most popular CI/CD tools for continuous integration and continuous delivery. With more than 16,000 stars and 6,500 forks on GitHub, Jenkins is being used by thousands of software companies. Its popularity comes from the fact that it is really good at what it does — building and releasing software pieces. You can refer to our comprehensive Jenkins tutorial for more information.

Core Concepts of Jenkins:

1. Project / Job

A Project or a Job is a collection of configurable instructions used for building developed source code, testing, collecting artifacts, sending notifications in emails, deploying in required environments, and much more.

2. Pipelines

A Jenkins pipeline is set of instructions and build steps configured in Jenkins to run all the processes in multiple environments in various steps.

3. Build Queue and Build Executor Status

Build Executor Status
Build Queue

4. Plugins

More than 1500 community-contributed plugins in Jenkins support building software varieties, automate tests, and deploy projects for production releases.

5. Master — Agent

Executes in remote parallel machines for shorter build and test running time using additional resources in the cloud.

The master node is usually the machine where Jenkins is configured to be running.
Agent nodes are generally the other machines that receive instructions from the Master node to help the executions run and finish faster in parallel.

How to Set Up Jenkins?

Before learning how to set up continuous integration Git Jenkins, you need to set up a Jenkins server in your system. If you already have the setup, you can skip this section and get right to continuous integration with Jenkins and Git. For now, let us walk through the steps by downloading, installing, and configuring Jenkins from scratch.

How to Download Jenkins?

Step 1: Download Jenkins

Go to the official Jenkins website and download Jenkins.

For example, if one has a Windows machine, a jenkins.msi file might start getting downloaded.

While in the case of macOS, one may initiate installing Jenkins via the Homebrew package manager by following the commands provided in instructions on the same page as mentioned above.

Note: In this blog, I have taken an example of installing Jenkins on macOS.

Step 2: Installing Jenkins on macOS.

Now, let us look at the process of installation of Jenkins locally on a macOS. Let us open the Terminal app in mac and run the below command and wait for the command to install all required tools on the mac machine successfully:

brew install jenkins-lts

Please note that the command shall work only when Homebrew is already pre-installed in the macOS machine. If not installed, kindly install Homebrew before running the above command in mac.

Step 3: Start Jenkins
Post successful installation of Jenkins, run the following command to start Jenkins:

brew services start jenkins-lts

Step 4: Configuring Jenkins as an Admin

Once Jenkins is successfully started, let us validate if the local Jenkins server is starting in the machine by navigating to the following URL in your browser:

http://localhost:8080

Suppose the “Unlock Jenkins” web page is displayed after navigating to the above-mentioned URL without an error message. In that case, the Jenkins server is confirmed to be up and running on the local machine.

Copy the above path for Admin Password highlighted in pink in the figure

Step 5: Execute the command.

Open Terminal app, and paste the above copied path as provided below:

sudo cat

Example:
sudo cat /Users/skhamaru/.jenkins/secrets/initialAdminPassword

Post successful execution of the above command, the initial admin password can be found displayed as shown in the figure below:

Step 6: Unlock Jenkins

Copy the newly found initial admin password from the terminal and paste it in the “Administrator password” field in the previously displayed Jenkins web page, as shown below.

Then, click on the ‘Continue‘ button.

Step 7: Customize Jenkins

Click on “Install suggested plugins” to install the most common Jenkins plugins.

Step 8: Create First Admin User

Once all the plugins are installed successfully, the first admin credentials will be created for the newly set up Jenkins server.

Enter valid credentials for the new admin for future configuration changes in Jenkins

Step 9: Instance Configuration.

Make sure that the below URL is as per configuration, post which “Save and Continue.”

Step 10: Complete the Setup

When the page below is displayed, click on the “Start using Jenkins” button and explore the newly set up Jenkins server’s user experience.

The newly logged-in admin page shall look as displayed below:

In this system testing tutorial, learn why System testing is important and all the intricacies of the System testing process.

Setting up Continuous Integration with Git Jenkins

While there are different ways for continuous integration with Jenkins and Git, let us explore the easiest way of integrating using GitHub’s “Personal access tokens” by following the below steps.

Configuring GitHub for Jenkins Continuous Integration

Step 1: Go to your Github profile and navigate to Settings.

Step 2: In the settings screen, click on the “Developer settings” menu and click on “Personal access tokens.”

Step 3: In the “Personal access tokens” tab, click on the “Generate new token” button and provide necessary details as desired (an example is provided in the below figure), and click on the “Generate token” button.

Step 4: Post successful creation of the token, the newly generated secret text in GitHub is to be copied for future usage in Jenkins.

Configuring Jenkins for GitHub Integration

Step 5: Next, go to Jenkins dashboard.

Click on “Manage Jenkins.”
Click on “Configure System” and go to the following ‘GitHub‘ section.

Note: If the above “GitHub Server” section is found missing in Jenkins, make sure to manually install the missing GitHub plugin from the installed list of tools shown below.

Steps to follow- Go to: Jenkins Dashboard -> Manage Jenkins -> Manage Plugins -> ‘Available’ tab -> Enter Git in search bar and filter -> Install required plugin

Step 6: Make sure to add the copied secret key in the above credentials by:

Click on the “Add” -> “Jenkins” button in the Github Server section’s Credentials field.
Select “Secret text” from the “Kind” dropdown.

Step 7: Paste the previously copied secret text from GitHub in the Secret field as displayed below, while providing an ID such as “jenkins-integration,” and then click on the “Add” button.

Step 8: Once the Secret text is successfully added, let us test the connection by clicking on the “Test connection” button and verify the confirmation message as highlighted below.

Creating Your First Jenkins Job Integrated Into a Test Project

Let us take a sample java project already integrated and configured with the Ant build tool to ease convenience. You can choose any sample project of yours if you like. Let us follow the below steps in Jenkins Dashboard to configure the same.

Step 1: Click on “New Item” in the Jenkins user interface dashboard on the left side.

Step 2: Let us start with creating a Freestyle project in Jenkins for ease of understanding for a beginner. However, we can choose to select any of the following:

Enter a project name in the requested textbox field and select the Freestyle project as shown on the image above.

Step 3: In the General section, check the field “GitHub project” and enter a valid Ant project as given below.

Step 4: Then, go to the “Source Code Management” section and enter the same link as above with an extra .git extension at the end of the URL, as shown in the below screen.

Step 5: Go to the “Build Environment” section and add the following steps shown in the image below. Make sure to add the command in the Targets field under the Ant Build step:

clean compile test package war

Step 6: Click on the “Apply” button and then click on the “Save” button.

Step 7: Once the new Jenkins configurations are successfully saved, the user shall be navigated back to the Project dashboard screen, where the user now clicks on the “Build Now” menu on the left-hand side of the displayed dashboard screen.

Step 8: As the Build has started to run, users shall be able to note the scheduled build running in the “Build History” section with a visibly blinking blue round icon. Clicking it will navigate the user to the respective “Build Status” page.

Users must click on the “Console Output” menu to walk through the build logs generated in the console while building in Jenkins.

Analyzing JUnit Test Results in Jenkins

Now that we have learned how to set up continuous integration with Git Jenkins and how to create the first Jenkins Continuous integration job within a test project, it is time to take it one step ahead. After creating a job with a Java project, you might also want to analyze the test results. This can also be done using Jenkins.

Let’s see how this can be done for JUnit.

Step 1: Make sure “JUnit Plugin” is already installed in Jenkins.
Go to Jenkins Dashboard -> Manage Jenkins -> Manage Plugins -> Click on the ‘Installed’ tab to view all the already installed plugins.

Step 2: Go to Project -> Click on ‘Configure’ -> Click on ‘Build’ tab.

Step 3: Click on the “Add post-build action” button at the bottom of the page.

Step 4: From the above list of items displayed in the dropdown, select the “Publish JUnit test result report” option.

Step 5: Add the generated XML test report XML path from the workspace in the Test report XMLs field as shown below.

Step 6: Click on the ‘Save’ and ‘Apply’ buttons.

Step 7: Click on the “Build Now” button and allow the scheduled build to complete.

Step 8: Observe that a “Test Result Trend” graph is generated on the project dashboard’s right side.

Step 9: On closer observation, we can observe that the x-axis **of the newly generated graph consists of Build Ids, and **the y-axis is composed of several test cases run in each build.

Step 10: Along with the generation of test result trend graphs, we can also observe that the “Latest Test Result” report is also generated in the dashboard to help users closely analyze the run tests status in each build.

Integrating LambdaTest Automation Plugin

Continuous integration with Jenkins and Git is a great way to streamline your test automation scripts. Furthermore, you can also implement a Jenkins Continuous integration to make your scripts easier to test. LambdaTest provides a Jenkins plugin to easily automate your Selenium test scripts by connecting your Jenkins CI instance to Selenium. If one follows the instructions as provided in the official website of LambdaTest, we can also observe that LambdaTest integration works smoothly to the above test run as well on any desired test environment.

I simply followed the steps provided on the official website to check how the integration works in Jenkins, and to my surprise, it worked like magic without any extra effort.

And the tests passed successfully in all the configured test environments!

A complete manual testing tutorial covering all aspects of Manual testing, including strategies and best practices.

Summary

There is still an ocean left in Jenkins to explore, learn, integrate varieties of plugins, configure builds in several combinations of languages and source code repositories and make the process better with the integration of pipelines in different stages and steps across projects.

Jenkins continuous integration is undoubtedly a one-step gateway for running quicker and automating robust ways to integrate the entire SDLC pipeline stages of build, test, and deploy effortlessly and efficiently by yourself at almost no extra cost. Now take your test scripts one step ahead with continuous integration with Jenkins and Git.

Thanks for reading this article! Leave a comment below if you have any questions. You can always post any questions or queries on LambdaTest Community for immediate assistance.

24 Testing Scenarios you should not automate with Selenium

Suparnakhamaru1 — Tue, 22 Nov 2022 11:41:35 +0000

While there is a huge demand and need to run Selenium Test Automation, the experts always suggest not to automate every possible test. Exhaustive Testing is not possible, and Automating everything is not sustainable.

Let us consider an example of Selenium WebDriver in this article and analyse what Selenium test scenarios are not greatly supported with it when used as a stand-alone tool. We will also see what all Selenium test scenarios are not best for automation.

Automated functional testing tests helps to ensure that your web app works as it was intended to. Learn more about functional tests, and how automating them can give you a faster release cycle!

Selenium WebDriver

Selenium WebDriver is a web automation tool, primarily works best across different browsers, browser versions, and desktop platforms (e.g. Windows, macOS, etc.). . However, if we want to automate specific test scenarios that were initially not intended to be supported with Selenium, we would be doing something wrong.

The agenda of this article is to create awareness of what Selenium test scenarios are NOT to be automated using Selenium WebDriver as a stand-alone tool.

1. Windows apps Testing

As Selenium WebDriver is specifically used for automated testing of web applications, we cannot use it for windows based applications.

For example, if we want to automate tests for a Windows application, such as a native ‘Calculator’, it is not possible via Selenium. We cannot even launch the Calculator app in the windows machine using Selenium or Appium commands.

Hence, Microsoft has been implementing and working on automating tests on Windows-based applications, which were otherwise not possible in Selenium and Appium for the last few years.

Today, Microsoft’s WinAppDriver works to a great extent when combined with the Selenium and Appium test framework on some of the native built-in apps of Windows 10 applications.

WinAppDriver sounds promising on Windows Apps Testing, yet there is a minor limitation here. It requires a C# codebase environment such as Visual Studio to run the first-level automation test scenarios, running on Windows apps before configuring it on a CI/CD tool like Jenkins.

Popular tools such as Sikuli and AutoIT are widely used to test applications of some Windows native components. that are often found integrated to an existing Selenium testing framework. This practice, in general, is not very helpful hence not much recommended for extensive usage.

We also have HP’s UFT, which is licensed and used in some windows-based application testing purposes. One needs to know Visual Basic (VB) scripting, as the tool does not have a diverse language support feature like Selenium.

2. Mac apps Testing

Similar to windows applications, Mac apps cannot be considered as Selenium test scenarios. However, Appium’s Mac driver can be integrated into an existing Selenium Test Framework to support a certain extent of basic automation on Mac applications.

As Appium Mac Driver is a layer written on top of XCUITest Framework, it would also be a better idea to go with Apple’s XCUITest Framework..

3. Native apps Testing for Mobiles/Tablets

Selenium automation testing is not suited for testing native mobile applications. Appium, when combined with Selenium, works fine on mobile applications.

Appium is a layer on top of UIAutomator, Espresso, and XCUITest frameworks. Hence, would be a better approach to go for the latter tools to achieve better efficiency and testability of maximum test scenarios.

4. Smart TV apps Testing

Selenium does not support testing of apps developed for Smart Televisions (TVs) It would most probably continue to be unsupported in the future, as Selenium test scenarios are meant for automated cross browser testing.

5. WatchOS apps Testing

As of date, automating tests developed for smartwatches (e.g. iOS, Tizen, etc.) is not possible with Selenium automation framework. There are hardly any options to realize this testing.

Undoubtedly, the same cannot be automated using Selenium anytime soon and shall remain unsupportive for some time to come.

6. Device / Hardware / Embed Testing

Selenium cannot be used to automate testing on Customised/Embedded Hardware..

Also, we always need to remember that if manual testing takes less time and effort, then automating those flows would be a bad idea, as automation test scenarios are usually considered for saving time, effort, and increasing reliability.

7. Captcha Verification

‘Captcha’ stands for “Completely Automated Public Turing test to tell Computers and Humans Apart”. Captcha is one of the challenge-response tests widely used in many websites to verify if the accessing end-user is a bot or an actual human being.

The standard practice while running automated test scenarios for Captcha, is to

Disable Captcha in test environments OR
Adding a hook to allow the tests to bypass the Captcha OR
Making Captcha field as an optional field ONLY in test environment, so that if the Captcha step is skipped, test remains unimpacted

However, Selenium test automation cannot be used to perform Captcha verification. These tests can only be realized by using one of the above practices that helps in bypassing (or disabling) Captcha.

Captcha has always been tricky for testers to automate & is possible under certain scenarios here a video for you to understand it better.

Through this usability testing tutorial, you will learn how usability testing is a great way to discover unexpected bugs, find what is unnecessary or unused before going any further, and have unbiased opinions from an outsider.

8. Barcode / QR Code Verification

We cannot read barcodes or QR codes using Selenium Web Driver. Barcodes and QR codes can be automated, but in reality, do not serve the real-time purpose to automation.

We have to integrate third-party APIs to the existing Selenium test scenarios for realizing automation of barcode/QR code verification. . However, it’s best if the test is performed in a manual manner as it yields higher accuracy.

9. Visual testing

Visual Testing (also referred to as Screenshot testing) is aimed to check the GUI and ensure that it is displayed as intended. You can capture full page screenshots with Selenium as well as capture screenshots of WebElements on the page.

However, visual Testing that primarily includes “Bitmap comparison” is significantly difficult to perform with Selenium Web Driver.

In a nutshell, capturing screenshots is not a hassle in Selenium Automation Testing but performing a ‘visual comparison’ of the images can be a tedious process.

10. 2FA — Two Factor Authentication

While it is possible to automate Two Factor Authentication mechanisms such as OTP validation, push notifications, etc. they are quite tedious and complicated to integrate with Selenium WebDriver. Only a limited part of the test process can be automated using the tool to run automated tests.

You can validate the OTP indirectly by tweaking and looking at the page content displayed to the end-users on the AUT, post submission of the OTP.

However, most organisations in the industry choose not to automate OTP submission & verification flow and prefer following one of the below test steps:

Disabling 2FA for specific test credentials that are used for running automated tests.
Disabling 2FA in Test Environments completely when automation tests are run.
Disabling tests based on test machine IPs (It may not be a good approach in larger organisations where test machines are randomly chosen from a huge list of machines).

11. Performance testing

Performance testing using Selenium Webdriver is not recommended. It is not like you cannot create Selenium test scenarios to run performance tests but Selenium is not optimized for the purpose of performance testing. Hence the entire exercise might not yield favorable results.

Well, WebDriver tests are subjected to many points of external and internal fragility which are beyond your control (e.g. browser startup speed, speed of HTTP servers, the response of third-party servers that host JavaScript or CSS, and the instrumentation penalty of the WebDriver implementation itself).

Instead, you can leverage tools that help you with front-end performance testing that provide accurate results. Tools like Google Lighthouse, LT Browser can help you run performance tests faster and more efficiently.

12. Security Testing

Web application security (or website security) is considered a top-most priority by enterprises since it protects the web app and other online services against different security threats. Hence, most enterprises have a dedicated team of Cyber Security experts who use their own set of tools and technologies to perform security testing.

Again, Selenium is not even counted in one of the tools through which security testing can be performed.Most in-demand tools used by cyber experts are JSky, Owasp Zap, Burp suite, etc.

13. API Testing

API testing is usually performed with tools like Postman, RestAssured, etc. . Selenium WebDriver is usually not used when testing at the API-level. .

It would be best to treat API testing as a separate testing type and run them separately in a different package/suite for achieving better visibility and efficiency.

14. Link spidering / Application Crawling

Conducting App Crawling helps find the broken links easily at no extra cost and time. This helps to find out if the app is behaving weirdly.

Many a time, link spidering is done in much easier steps and it hardly requires much time or effort to write the test scripts for the execution of the same.

While there are several easier ways of conducting link spidering tests in apps, it is suggested not to use Selenium WebDriver to crawl into all the links in the apps. The primary reason is that tasks like finding broken links with Selenium WebDriver are suited when there are less number of links on the page (or website).

Hence, you should choose tools for link spidering based on your requirements.

15. Email Integrations and Social Network Logins

Another case where you should not automate Selenium test scenarios through UI is email and social media logins. Ideally doing this isn’t considered one of the best practices for automation. It is recommended to check the usage guidelines when performing automation else it might lead to serious consequences (including account shutdown).

By integrating the above apps on the UI test layer might lead to test flakiness along with increased test maintenance.

Hence, it is encouraged to use the API layers. By using the login and emailing features, the chances of breaking the API are comparatively lower. The end result is lower test flakiness and faster test execution.

16. Audio / Video streaming

Consuming video content on YouTube and popular OTT platforms have become a norm. However, top-notch user experience can only be dished out if the product is thoroughly tested by the product development and testing teams.

Testing or automating audio/video streaming on a large scale is still considered one of the biggest challenges. There are tweaks of trying to test it remotely by passing the URLs in sites such as StreamTest but the test result may not be very reliable or assuring to count on.

While we are reaching a point to solve this problem via AI with better validation and verification of video quality to the end-users, it is not possible to automate with Selenium test scenarios as of date.

17. Screen Sharing platforms

As remote working became the new normal since 2020, several online meeting platforms such as Zoom, Google Meet, etc. witnessed a surge in demand.

With such tools in use on a large scale, testing and automating user test scenarios becomes much more critical. However, on closer inspection of the web pages of such platforms, we notice that tools like Selenium might not bring much-needed efficiency to the testing approach.

Front end of most meeting apps do not meet the basic automatable criteria and might require tools more sophisticated than Selenium.

As a result, many of these platforms collect feedback from their end-users to optimize the product features as per the feedback.

18. Image Verification

Selenium Webdriver alone cannot be used for comparing expected and actual images. Third-party tools like AShot when integrated with existing Selenium Test Framework can help in achieving that task. However, the integration provides the flexibility to compare images to a minimal extent (i.e. comparison accuracy might not be that great)

Although AShot cannot be used wholeheartedly across all the tests, it could be a boon in testing a few critical scenarios.

Several open-source snapshot testing tools such as Uber’s snapshot test framework for iOS and Facebook’s Screenshot test for Android are popularly used across mobile-first projects.

19. Layout Testing

Selenium can capture effective screenshots and save them in desired locations, as per our convenience across tests. AI-based comparison of the screenshots against expected screenshots is still not a built-in feature of Selenium.

One particular tool which is quite popular among Layout testing tools is Galen Framework. It is widely used in responsive web design testing of websites and apps.

Tools like LT Browser make responsive testing faster and easier by providing 45+ screen resolutions for different mobiles, desktops, and tablets. Using LT Browser you can streamline UI/UX development, test content placement, layout alignments, and assess visually heavy websites on different screens side-by-side. Further, you can generate performance reports of your websites and simulate network conditions to see how your website looks and behaves.

20. File Upload and Verification

We can go ahead and run Selenium test scenarios in the web application where Selenium Webdriver can be used. For example, you can automate to mimic the action of a user clicking on the upload button (which can upload content such as an image, file or a video from the local system).

However, post successful uploading action, we cannot validate using Selenium to verify if the file is successfully uploaded and displayed in the UI. We also need to rely on third-party tools like AutoIT to upload the contents from the local system for the Windows machine.

And post upload, we may have to integrate it with another image verification tool to test that the image is successfully uploaded. This might sound impractical but is doable only if a business wants to automate the flow at any cost. Interestingly, human Intervention cannot be avoided if tests are flaky and unreliable.

Do you know how to upload and download a file in Selenium? Watch this video to learn how to upload and download files in Selenium WebDriver using different techniques.

21. File Download and Verification

While Selenium test scenarios can be used in performing the action to download a file/content from the web to the local system, it cannot validate the ongoing progress of the downloading content in the Downloads efficiently. The only way to verify if the file is downloaded is by comparing the number of items in the downloaded folder (before and after the download process).

The usage of this feature is still limited and needs further improvements to help and conduct effective Selenium Test Automation.

One simpler way is to use the HtmlUnit driver to download any attachments if one can’t avoid skipping it. Also, verifying if the downloaded content matches with the expected contents of the design specifications also requires human intervention to a certain extent.

22. Exploratory Testing

The speciality of Exploratory Testing is completely based on human experience, instincts, and observation while exploring the app as an end-user.

Ideally, Exploratory Testing should not be automated using any Selenium test scenarios or any other automation tool. While exploratory testing is a recommended testing step in any release cycle, we should avoid trying to automate steps from Exploratory testing. It is best to perform exploratory testing using a manual approach under any given situation.

23. Usability Testing and Accessibility Testing

Identifying tools and resources is critical in testing the app’s right UX and functional features from the end-users perspective. However, performing Usability Testing at scale can turn out to be a costly and time-consuming exercise.

It does not matter how bug-free a built app is, and how efficient and fast the app performs if at the end of app release, the end-users find the app difficult to use or find is unusable due to poor UI/UX. Factors like efficiency, reliability, and impressive UI/UX go a long way in building a winning app experience.

Accessibility Testing is a specialised form of testing that makes sure that the Application Under Test (AUT) is fit for use by the differently abled audience. Hence, accessibility testing is treated as one of the forms of usability testing.

This testing type is also not the right tool to be used with Selenium WebDriver to automate Selenium test scenarios. Instead, you should opt for tools like Wave, Taw etc. This is a special form of testing, hence it is recommended to go for tools that test the applications particularly from an accessibility point of view.

24. Libraries and Frameworks Testing

Most libraries and frameworks are functional in nature and require strict functional testing to be performed on them to achieve efficient and reliable test coverage.

As Selenium Webdriver is a UI testing tool for Websites (or web apps), it would not be a good idea to use it for testing the built-in UI-less libraries and frameworks. Hence, you should look for better tools to test libraries and frameworks.

Most teams may not rely on any third-party tools for simplicity. Most teams would like to automate tests and the developed source code itself using the development language and framework itself with minimal usage of testing frameworks such as Junit, Nunit, TestNG etc.

Black box testing? Don’t worry; we will be covering what is Blackbox testing, merits, demerits, types & techniques.

Wrapping it up!

Well these are the 25 scenarios where performing test automation won’t be helpful to you. However there are chances that in some cases it might work, but you can refer to this blog and understand the reasons why it’s advisable to not automate these test scenarios. Perform your free end to end automation testing at a blazing speed!!!

Also, we need to constantly remind ourselves that automation is to help us save time, money and efforts. But one needs to have an understanding of Selenium’s core usage is browser compatibility testing.

Suppose any automated script time or execution time is exceeding the time taken to run manually. In that case, the same may not be the best scenario to automate in the real world, unless the automated work reduces the time in the long run effectively or helps business reach the market fast with quicker & reliable releases. You just cannot use Selenium everywhere!

Just how exploratory testing is impossible, automating everything is also a bad move in many cases. Prioritising what tests to automate first and how quickly to integrate the created automated tests to CI-CD pipelines to add value in the development lifecycle is way more important than automating each and every use case in the project. I hope this article helps you understand why you shouldn’t automate everything using Selenium. Feel free to reach out if you have any questions or feedback.

Happy Testing!

Complete Guide To Selenium Testing with GitHub Actions

Suparnakhamaru1 — Tue, 22 Nov 2022 09:25:05 +0000

CI/CD pipelines are here to stay and contribute tremendously to continuous integration and delivery across all global projects. This article will be a guide to configure, set up builds and tests with “GitHub Actions”, primarily using Selenium WebDriver. This article shall also cover some of the most generic GitHub Actions examples, and user flows.

You can accelerate your test cycles and trim down your overall test execution by running Selenium tests in parallel from GitHub Actions on LambdaTest cloud.

Let’s get right into it, starting with an intro to Selenium GitHub Actions!

In this Appium Automation tutorial, learn about Appium and its benefits for mobile automation testing. Take a look at how Appium works and see how to perform Appium testing of your mobile applications.

Introduction to GitHub Actions

GitHub Actions is one of the most recent CI workflow mechanisms that is looking not only impactful and promising but also quick and easy. The configurations and set-up are found to be within the Git repository pages. It’s now easier than ever to automate your software workflows, thanks to GitHub Actions, including CI/CD. As we no longer need to go to a third-party site to peek on our build runs, it’s going to save a lot of our otherwise spent time. To get started with Selenium test automation, we must first understand the following GitHub Actions example.

GitHub Actions Component

If we visit any GitHub repository project, we will notice a series of menu tabs, as shown in the below sample figure.

The red highlighted box in the above figure is the section of our interest, where lies different varieties of GitHub workflows and GitHub Actions components.

Attached below is a GitHub Actions tab screen figure when sample workflows are set up and running.

Like the usual CI mechanism, if the build passes, then we see a Green Tick Mark (✅). If the build fails, then we see a Red Cross Mark (❌) beside the particular run, which is usually auto-initiated as soon as the user commits any change in the GitHub repository.

GitHub Actions tab primarily provides us with a mechanism to build multiple workflows as per the instructions or commands provided in YAML files, placed in .github/workflows folder of the GitHub repository, as can be observed in the figure below.

The GitHub Actions tab screen also provides the flexibility to filter build runs as per our need with few basic filtering options. We can either search or use the filtering options directly.

Provided below is an image of a sample scenario where the user views the filtered view of the run builds triggered on every git push that passed successfully. We can observe that build run time is also displayed on the right side of each ran build in the displayed workflow view.

Creating GitHub Actions Workflow

When a user visits the GitHub Actions tab for the first time, when no configurations are done yet, we will notice a view attached in the screen below.

We may also notice that GitHub is auto-suggesting users to set up the possible workflow to be selected at the top of the page.

Example:

If the project is a Swift project, automatically, the user is suggested to set up a “Swift” workflow at the top of the suggestions list, as we would notice below shortly.

If the project is a Maven-based Java project, automatically, a user is suggested to set up the “Java with Maven” workflow at the top of the suggestions list.

Suppose a user selects any of the above-listed options based on the project’s need. In that case, the user is already automatically provided with auto-generated scripts in a YAML file, which can be further configured as per our need and then committed into the existing GitHub repository.

Once the YAML file is committed & pushed to GitHub repo with required commands/instructions, we would start noticing that the workflow is triggered automatically. The build starts running remotely on the same GitHub Actions tab. We can see the status of the build run as well on the same screen.

Now, let us create a new workflow and see how it works below:

Step 1: Let us consider a Sample Test repository of an XCUITest Swift project.

Step 2: Click on the “Actions” tab of the XCUITest GitHub repository page.

Step 3: When no workflows are set up yet, we would most probably notice that the GitHub Actions page already auto-suggests us to set up a recommended workflow at the top of the page, as can be found from the screenshot below.

Step 4: Click on the “Set up this workflow” button in the required workflow, which we want to set up. In this case, it would be the Swift workflow, as our chosen sample XCUITest project is a Swift based project.

Step 5: Immediately, we would notice that a list of predefined commands is automatically generated, as shown below.

While a slight modification would be required to build any set-up, the basic commands are auto-generated by GitHub’s initial sample set up, which we can modify based on our project’s need further.

Step 6: On the right side of the page, we would notice the “Start commit” button.

Step 7: Clicking on the “Start commit” button allows the user not only to add comments for commit, but also lets the user to:

Directly commit to master / main, which is selected by default, as can be observed from the figure below.
Create a new branch for the user to create a new pull request.

Step 8: Once the commit is successfully done, we would notice that our new workflow is successfully created in the “Actions” tab, where initially the workflow’s build job goes into a Queue.

Step 9: Post Queue, the workflow’s build job starts building and running the instructions/commands in the YAML file committed by us and changes the build status from “Queued” to “In progress“.

Step 10: Post completion of all the build runs, we would notice the status changed to pass/fail based on the outcome of the build run.

If the build passes, then we see a Green Tick Mark (✅) beside the built run.

If the build fails, then we see a Red Cross Mark (❌) beside the built run.

Test website on different browsers on a cloud platform. Get instant access of Browsers like Chrome, Safari, Firefox, Opera, Yandex, Brave, 3000+ desktop & mobile environments. Try for free!

Workflow File Structure

One GitHub Repository may have multiple workflows configured, each building different actions required for building, testing, and deploying. The workflow file structure looks very similar to the Jenkins-file and CircleCI config (YAML) file formats. So, the formats we need to keep in our mind while dealing with YAML files are that we cannot enter incorrect formats, and we need to follow the proper indentation, colons and dashes all across the file, which primarily consists of key-value pairs, lists and maps.

Below is an example of a sample skeleton auto-generated when the project’s workflow is selected as “Java with Maven” based.

When we look closely at the syntaxes of the commands used in the above Yaml file, the following shall be observed-

Name of workflow
On Attribute
Jobs
Job name
Job runner information
Job Steps

Let us explore all these one by one.

How to name your Workflow?

If we want to have multiple workflows to build a CI pipeline to run tests with Selenium GitHub Actions, it is recommended to name your workflows accordingly. It is also to be noted that it is not a mandatory field, and if skipped, GitHub shall create a default name for us from the defined path & name of the file.

In the above-configured YAML file, line 4, key — ‘name’ denotes the name of the Workflow, the value of which shall be displayed in the ‘Actions’ tab, post configuration as shown below.

When to trigger the Workflow?

‘On’ attribute is mandatory and is seen as an important step to be added in every workflow as this step allows the user to define when to run the workflow in a GitHub Actions example.

There are different approaches to trigger the workflow, such as:

push
pull_request
release

…and many more.

We can also use the above events in different ways with Selenium GitHub Actions.

Where do the Build Jobs run?

Based on the user-chosen build machine configuration, primarily from the below list of predefined GitHub supported VMs, all GitHub jobs run in the Actions tab.

In our example, we have chosen “macos-latest” to run our build from the above list of GitHub supported VMs. We can alternatively use any of the other VM with required specific OS versions as well.

How to configure steps for the Build Runs?

Let us consider a sample example of a Maven & Java based Test project.

Jobs and Job Name

In the above workflow sample, our jobname is “build” in line 13, right under ‘jobs’. We also name it “maven_build_job” if that lets the user know better about the job.

We should note that:

Every workflow must have at least one job.
All workflow jobs run in parallel by default.

Each job may have multiple job steps.
If any job needs to be run in sequential order like in a CI pipeline, where the test job needs to run after completion of the build job, we can add “needs” dependency in the test job.

As a result of which, the same pipeline status can be viewed when a PR is raised, as shown below:

Job Steps

It consists of a List of defined commands that perform certain actions.

Each Step may have the following parameters:
name -> optional
uses -> defines “what action to use” and “location of action”.
runs -> one or series of commands to run in the Job Step in OS’ default shell

An example of a Job step can be observed below:

How do Pull Requests reflect on the Build Runs?

All the latest build runs of GitHub commits in PR are automatically linked to the below-marked section. Clicking on the “Details” link also navigates the user to respective builds.

A sample view of a build is provided below when a user clicks on the Details link in the Build run section of Pull Request.

We can also click on each of the above accordions to view the detailed logs of the build runs step by step.

One such accordion is expanded below for our understanding. We can click and expand multiple accordions if required, based on our convenience.

How to manage your Workflow runs?

While we can also build CI pipelines using Selenium GitHub Actions, there is much more we can do from what we have discussed in this article, considering all GitHub Actions examples.

Some of them are listed down:

Deleting a workflow

Disable a workflow

Clicking on the below marked “Disable workflow” button in the workflow build screen disables the selected workflow from the Actions tab.

Cancelling a running workflow

If a workflow is currently running, we can cancel the build run by clicking on the “Cancel workflow” button highlighted below.

Post cancellation of the running workflow, we would notice the below screen.

Re-running a workflow

Manual click of the “Re-run jobs” button as marked in the workflow screen re-runs the selected build again.

Creating Status badges

If we view any build run in Actions tab, we would notice three-dots on the right side of the screen as shown in the image below, which is selected in blue below “Re-run jobs” dropdown.

On clicking above “…” button, we click on “Create status badge”, copy the content and paste it in our desired location such as in the readme file to denote the latest build run status as shown below in the red highlighted section.

How does the Build Run in ‘Actions’ workflow when triggered?

Upon completing the successful build run of the commit change, we can also observe and analyze the step-by-step build run as we see in any other CI-CD tool, as shown below.

You can accelerate your test cycles and trim down your overall test execution by running Selenium tests in parallel from GitHub Actions on LambdaTest cloud.

Executing Selenium Testing Scripts with GitHub Actions Workflow

We hope all the GitHub Actions examples above have helped you understand the workflow better. Now, let us configure a workflow by applying all the above GitHub concepts we learned so far with Selenium test automation..

Step 1: Sample Selenium Test Script

Write a sample test class in your sample Maven Java project, integrated with Junit under the folder src/test/java.

package SeleniumProject.Maven;

import java.util.concurrent.TimeUnit;
    import org.junit.Test;
    import org.openqa.selenium.WebDriver;
    import org.openqa.selenium.chrome.ChromeDriver;
    import org.openqa.selenium.chrome.ChromeOptions;
    import io.github.bonigarcia.wdm.WebDriverManager;

    public class AppTest 
    { 
     public WebDriver driver; 
     public String baseUrl = "https://www.lambdatest.com/";  

     @Test             
     public void test1() {      

      WebDriverManager.chromedriver().setup();
      ChromeOptions options = new ChromeOptions();
      options.addArguments("--no-sandbox");
      options.addArguments("--disable-dev-shm-usage");
      options.addArguments("--headless");
      driver = new ChromeDriver(options);

      driver.manage().timeouts().implicitlyWait(20,TimeUnit.SECONDS);  
      driver.manage().window().maximize();  
      driver.get(baseUrl);
      driver.close();
     }     
    }

Step 2: Adding dependencies

Make sure to add the mentioned below required dependencies in pom.xml file

 <dependencies>
        <dependency>
          <groupId>junit</groupId>
          <artifactId>junit</artifactId>
          <version>4.11</version>
          <scope>test</scope>
        </dependency>
        <dependency>  
     <groupId>org.seleniumhq.selenium</groupId>  
     <artifactId>selenium-java</artifactId>  
     <version>3.141.59</version>  
        </dependency>  
        <dependency>
     <groupId>io.github.bonigarcia</groupId>
     <artifactId>webdrivermanager</artifactId>
     <version>4.3.1</version>
        </dependency>
      </dependencies>

Step 3: Workflow Configuration

While adding a new workflow, make sure to add the below YAML file lines as provided below.
If we note closely, there is no significant difference between test_1 and test_2, as only the maven test command is run differently.

  name: Build and Test

    on: [push]

    jobs:
      build:
        runs-on: ubuntu-latest
        steps:
        - uses: actions/checkout@v2
        - name: Set up JDK 1.8
          uses: actions/setup-java@v1
          with:
            java-version: 1.8
        - name: Build with Maven
          run: mvn -B package --file pom.xml
      test_1:
        runs-on: ubuntu-latest
        steps:
        - uses: actions/checkout@v2
        - name: Set up JDK 1.8
          uses: actions/setup-java@v1
          with:
            java-version: 1.8
        - name: Test with Maven
          run: mvn clean test
      test_2:
        runs-on: ubuntu-latest
        steps:
        - uses: actions/checkout@v2
        - name: Set up JDK 1.8
          uses: actions/setup-java@v1
          with:
            java-version: 1.8
        - name: Test with Maven
          run: mvn -B package --file pom.xml test

Step 4: Test Executions in GitHub Actions Workflow

Post running of all workflows in the GitHub Actions tab, we should notice the outcome/status of all Build runs in the below format.

As we can notice below, the builds have run successfully, but each of the test jobs also has run successfully in a few seconds.

Step 5: If there is an error of Google Chrome not installed in the configured VM, we may have to add the below configuration in the above workflow Yaml file. Although, this may be an optional step.

  jobs:
      build:

        runs-on: ubuntu-latest

        steps:
        - uses: actions/checkout@v2
        - name: Set up JDK 1.8
          uses: actions/setup-java@v1
          with:
            java-version: 1.8
            server-id: github 
            settings-path: ${{ github.workspace }} 
        - name:  Install Google Chrome 
          run:   |
                  chmod +x ./scripts/InstallChrome.sh
                              ./scripts/InstallChrome.sh 
        - name: Build with Maven
          run: mvn -B package --file pom.xml test

In the above lines of code, we may also notice that on the Step Install Google Chrome: multiple commands are passed for running. This syntax is to be used whenever we want to pass multiple commands in a single build step run.

Executing Selenium Test Automation with LambdaTest and GitHub Actions

In order to perform Selenium GitHub Actions using LambdaTest, we need to set it up by following the instructions provided on the official website. Once you have done that, we can start your first LambdaTest tunnel using our LambdaTest credentials in the Terminal/Command Prompt app of your system.

Once the tunnel is started, let us try to test it after integrating the Tunnel to Github Actions by passing the following commands in the GitHub Actions’ YAML file:


    name: Java CI with Maven Test run

    on: [push]

    jobs:  
      test-in-LT-tunnel:
        runs-on: ubuntu-latest
        steps:
        - name: Start Tunnel
          uses: LambdaTest/LambdaTest-tunnel-action@v1
          id: tunnel-id
          with:
             user: suparna3107@gmail.com
             accessKey: add your own access key here..
             tunnelName: "testTunnel-id"
        - uses: actions/checkout@v2
        - name: Set up JDK 1.8
          uses: actions/setup-java@v1
          with:
            java-version: 1.8
        - name: Build with Maven
          run: mvn clean test

The above commands should start following all the passed instructions, and we should see a screen such as shown below in the Github Actions tab for the newly committed code.

To ensure that there are multiple tunnel commands in multiple files, we need to provide unique tunnel ID names to each command run so that our tests or builds do not fail due to the same tunnel name copied everywhere.

When the build job is run successfully, The build job turns green, and we can see that all the build job steps have passed successfully, as shown below.

New Selenium IDE supports cross browser testing and parallel tests for automation along with record and replay function. Here’s why you shouldn’t miss out on it!

That’s All, Folks!

Congratulations! We have just created a series of CI workflows for executing Selenium tests with Java and Maven. Github Actions has made CI look much more straightforward than ever before, right?
Similarly, the above approaches can be used for building and running all other tests as well just by changing the run command in job steps where required. This is a crucial step to perform Selenium test automation with GitHub Actions. Run your Selenium tests in GitHub Actions.

Feel free to explore the sample project here to explore and have further hands-on experience with Selenium GitHub Actions.

Happy testing!