Introduction
In modern software development, web automation has emerged as a vital component, particularly in the context of organizational applications. It has become increasingly clear that test automation plays a significant role in speeding up software delivery cycles for almost every organization. The significance of test automation to organizations is supported by the findings of an extensive poll conducted by Simform, which involved the participation of more than 1500 individuals.
The survey encompassed a diverse range of roles within the software development and testing domains. The statistical breakdown reveals that 24.5% of the participants identified themselves as software developers, while 18.4% were automation testers with others being manual testers.
The report's image credit goes to Simform.
Reducing the time required for software releases stands out as the most commonly cited reason for embracing test automation.
However, this is merely the tip of the iceberg, as there are numerous evident advantages of incorporating test automation into organizations' transition from traditional to agile software development methodologies.
In recent years, Agile and DevOps have evolved from mere buzzwords to rigorous practices adopted by a majority of organizations. Test automation has become a crucial step to keep up with the rapid speed of software releases as agile development's fast-paced and collaborative nature has become a prerequisite in contemporary software development.
Extensive research and hands-on experience have demonstrated that incorporating test automation is a powerful strategy for guaranteeing the dependability and scalability of software applications. Organizations can benefit from test automation features and advantages provided by platforms like LambdaTest such as distributed test execution, real-time web testing, parallel platform testing, and AI-powered test, among others.
With the increasing complexity of web applications, manual testing becomes time-consuming and error-prone. In the world of web test automation, a plethora of tools exist, among them renowned frameworks like WebdriverIO and Playwright.
Playwright, released by Microsoft in 2020, has been gaining popularity and seeing rising adoption, as indicated by the 2022 State of JavaScript survey.
For software developers and testers involved in web automation projects, the decision of selecting the most suitable tool for the job is of utmost importance. This decision holds immense significance in ensuring project success. Numerous options for test automation tools and frameworks are readily accessible, including well-known names like Cypress, Selenium, Playwright, WebdriverIO, Puppeteer, and even Appium, among several others.
In this post, you will explore what Playwright and WebdriverIO are, the differences between them, their features, what makes one stand out from the other, how they automate web tests, what programming languages they support, and why it's essential to choose the right tool for web automation projects.
Overview of Playwright
Playwright enables you to create automation scripts that interact with web pages in any web browser, including Google Chrome, Apple Safari, and Mozilla Firefox. Playwright was developed by Microsoft.
It is fundamentally a Node.js package that automates Chrome, Firefox, and WebKit using a single API.
Because of its speed and flexibility, it is quickly gaining popularity among the developer and tester communities. It increased from 19% in 2020 to 34% in 2021, with a continuous increase even this year.
Overview of WebdriverIO
WebdriverIO is a popular full-featured web automation framework built on Node.js.
Its main objective is to enable smooth web testing automation, adhering to the WebDriver protocol.
With a comprehensive set of APIs and plugins, WebdriverIO effortlessly integrates with well-known testing frameworks like Mocha and Jasmine.
This seamless integration enhances the framework's abilities and adaptability to meet various testing requirements.
According to the WebdriverIO official website,
WebdriverIO is a progressive automation framework built to automate modern web and mobile applications. It simplifies the interaction with your app and provides a set of plugins that help you create a scalable, robust, and stable test suite.
Progressive automation is part of the automation methods where test modules are tested one after the other. This entails writing automated test scripts simultaneously with the development code for faster testing and identification of problems along with quick fixes.
Why You Should Choose the Right Tool for Web Automation Projects
The success of your project can be greatly impacted by selecting the appropriate tool for web automation initiatives. A tool that is not suited for the task at hand can lead to a decrease in productivity, quality, and test coverage.
For instance, Playwright is faster compared to WebdriverIO and provides a more consistent testing experience across multiple browser platforms. It has features like video and trace logging, which can be helpful when debugging issues.
On the other hand, WebdriverIO provides a more mature framework for web automation projects. It has extensive documentation and features like video and trace logging are supported with third-party package.
What is Playwright?
Playwright is a Node.js package offering a high-level API for automating modern web apps with ease on several browsers, including Chrome, Edge, Firefox, and Safari.
It was developed by Microsoft in 2020 to enhance the speed, stability, and reliability of web test automation.
How Playwright Automates Web Tests
Playwright automates web tests by allowing you to simulate user interactions, such as clicks, scrolls, and keyboard inputs, and to perform complex actions like filling out forms and navigating between pages. It also has built-in support for waiting for specific page elements to appear before proceeding with the test, which helps to make tests more stable and less prone to flakiness.
Languages Supported by Playwright
Playwright is designed to accommodate multiple programming languages, giving users the flexibility to work with JavaScript, TypeScript, Python, C#, and Java.
Playwright is a flexible tool that can be utilized by a variety of developers regardless of programming language because it supports multiple languages.
How Playwright Works
Playwright uses API to automate its supported browsers. It works on the principle of 3 main core concepts: Browser, Context, and Page.
Browser: The first step in conducting tests is to open a browser. Playwright accomplishes this using a Browser class object, which is nothing more than a Chromium, Firefox, or Webkit instance.
Context: Playwright uses browser contexts to enable parallelization. A "browser context" is an isolated incognito-like session within a browser instance.
Within a context, a page is a brand-new tab or pop-up window. This object will be the target of every test activity.
Playwright launches a browser session in headless mode, whereby you don't get to see the browser window. Instead, Playwright navigates to the specified URL and carries out the test actions such as making assertions or clicking a button.
Playwright makes all communication requests through a single WebSocket connection. This enhances the performance of your tests and reduces errors by allowing commands to be sent over a single WebSocket connection.
Browsers Supported by Playwright
Playwright has comprehensive support for all the latest web browsers, such as Chrome, Firefox, Safari, and Edge. Additionally, it accommodates the Microsoft Edge browser and Chrome's headless version, both built on the open-source Chromium platform.
It is important to test your application across multiple browsers to ensure consistent behavior across multiple browsers because different browsers render pages differently.
Playwright offers a cohesive API that allows seamless interaction with web browsers. This advantage means you can write tests just once and execute them across various browsers without the need to create separate codes for each. If you're interested in learning Playwright in 6 hours, there's a video available on YouTube: Video to learn Playwright in 6 hours.
How to Set Up Playwright in Your Project: Walk-Through
Before proceeding, make sure to meet the following requirement:
Have the latest NodeJS installed on your local machine.
-
Confirm the Installation: Run the following command in your terminal to confirm a successful NodeJS installation:
node -v. You should see something similar to this:$ node -v v18.15.0 -
Confirm a successful NPM installation: To verify that the NPM installation was successful, execute the command
npm -v. When you install NodeJS, NPM is automatically included:$ npm -v 9.5.0 Download and install VSCode if you are not already using the code editor.
Next, you will need to install the Playwright VSCode extension to make testing easy.
Install Playwright:
Open VSCode and hit CTRL + SHIFT + P to open the command pallet.
Next type Install Playwright and hit Enter.
- Upon successful installation, you should see in your terminal that you have successfully installed Playwright
- This installation comes with Chrome, Firefox, and WebKit browsers installed and some bootstrapped test files in the
testsfolder.
// tests/example.spec.ts
import { test, expect } from '@playwright/test';
test('has title', async ({ page }) => {
await page.goto('https://playwright.dev/');
// Expect a title "to contain" a substring.
await expect(page).toHaveTitle(/Playwright/);
});
test('get started link', async ({ page }) => {
await page.goto('https://playwright.dev/');
// Click the get started link.
await page.getByRole('link', { name: 'Get started' }).click();
// Expects the URL to contain intro.
await expect(page).toHaveURL(/.*intro/);
});
Writing Your Playwright Test: Code Snippet Examples
const playwright = require('playwright');
(async () => {
for (const browserType of [playwright.chromium, playwright.firefox, playwright.webkit]) {
const browser = await browserType.launch();
const context = await browser.newContext();
const page = await context.newPage();
await page.goto('https://lambdatest.com');
await page.screenshot({ path: `example-${browserType.name()}.png` });
await browser.close();
}
})();
In this code snippet example, we navigate to lambdatest.com in Chromium, Firefox, and WebKit, and save 3 screenshots of the LambdaTest website.
Top Features of Playwright
Playwright.dev comes with outstanding features among with are the following:
- Ability to handle scenarios that cover multiple pages, domains, and iframes support,
- Supports automatically waiting for elements to be ready before executing actions (e.g., clicking on elements, filling out forms).
- Supports interception of network activity for stubbing and mocking of network requests.
- Ability to emulate mobile devices, geolocation, and even permissions.
- Support for native input events for mouse and keyboard.
- Supports file uploads and downloads.
Playwright.dev is highly versatile and can cater to several use cases.
Multi-Browser Support
Playwright supports multiple web browsers:
- Google Chrome,
- Microsoft Edge,
- Mozilla Firefox,
- Apple Safari for desktop and mobile devices.
Because of the support for multiple browsers, you can create a single test script that will work flawlessly across various platforms and browsers.
Cross-Browser Testing
Playwright allows you to test web applications across different browsers simultaneously using a single test script. This feature saves you a lot of time and work because you no longer need to create distinct test scripts for each browser. It supports all major browsers as I have already mentioned.
Speed and Reliable
Playwright is designed with speed and reliability in mind. Tests written with Playwright can run simultaneously across several browsers, leading to a shorter test execution time.
Headless Execution
Playwright supports both headless and non-headless execution modes. Running tests in headless mode means that the browser GUI is not displayed while the tests are being executed, which enhances the speed and efficiency of test execution.
However, the non-headless mode enables developers and testers to step through each test case visually.
Network Interception and Mocking
Playwright also allows network interception and mocking, allowing you to intercept and modify network requests and responses programmatically. This feature is highly useful in simulating different network scenarios and testing more effectively.
Mobile Emulation
Playwright allows developers and testers to test mobile versions of web applications by emulating mobile devices within browser windows. This is done through the use of device emulation profiles, which can replicate features such as screen size, touchscreen inputs, and sensors like GPS.
Native Events
Playwright supports native event simulation for mouse and keyboard user input events, making it more reliable as it automates user behavior. This feature helps prevent issues like test-only failures, where the test passes when run alone but fails when run with other tests.
Video Recording
Playwright.dev can record videos of test sessions for easier debugging of issues and improved collaboration. This feature allows you to view and share the exact steps that were taken during testing, leading to more efficient communication of issues.
Tracing and Debugging
Playwright includes built-in tracing and debugging tools that allow developers and testers to better understand how code executes. Tracing your application helps track down performance problems and availability while debugging in Playwright enables you to inspect, modify and set breakpoints in your test code.
What is WebdriverIO?
WebdriverIO is an open-source test automation framework and a Node.js library that provides a high-level API for automating web applications across multiple browsers. Its main goal is to make online test automation easier to use, even for developers with little prior test automation experience.
WebdriverIO Architecture: How It Automates Web Tests
Because WebdriverIO automates web tests using the WebDriver protocol, it can communicate with any browser that adheres to the standard, including Chrome, Firefox, and Safari. It also uses a testing framework, such as Mocha or Jasmine, to organize tests and generate reports.
Languages Supported by WebdriverIO
Numerous programming languages, including JavaScript, TypeScript, Java, Python, C#, Ruby, and many more, are supported by WebdriverIO. It is a flexible tool that developers from all linguistic backgrounds can utilize as a result.
How WebdriverIO Works
WebdriverIO is built on top of NodeJS - an open-source framework that is frequently used for application development, and is packaged into npm. This is the foundation upon which WebdriverIO is based.
In this section, I will walk you through the installation of WDIO and how to write your first test cases using the WDIO framework.
How to Set Up WebdriverIO in Your Project: Walk-Through
Follow the numbered step-by-step guide below to get started with WDIO:
- You need a blink-based browser, e.g. Chrome
- Optional: Browser Driver, e.g. ChromeDriver
Install NodeJS as follows: You need at least NodeJS v12 to be able to get started with WebDriverIO. Download and install the latest version of Node from https://nodejs.org/en/download/.
Install NodeJS: Follow the installation guideline to install NodeJS.
-
Confirm Installation: Run the following command in your terminal to confirm a successful NodeJS installation:
node -v. You should see something similar to this:$ node -v v18.15.0 -
Confirm a successful NPM installation: To verify that the NPM installation was successful, execute the command
npm -v. When you install NodeJS, NPM is automatically included:$ npm -v 9.5.0 -
Create A Project Folder: A project folder would help you organize your project data inside a single folder. To create this folder, run this command:
mkdir webdriver-testThis would create a folder called
webdriver-test. -
Change Folder: Change your folder into the newly created
webdriver-testby running this command:cd webdriver-test
-
Create a
package.jsonfile: To start a new project, open your terminal in the same folder aswebdriver-testand type this command:npm init -y
-
Install the WebdriverIO CLI tool: After having initialized your project, you now need to install the WebdriverIO CLI tool.
Run this command to do that:npm install @wdio/cliThis would then install some dependencies like the core framework itself and the WDIO CLI.
-
Generate a Config File: The config file contains all the necessary information required to run your test suite. The config file is a NodeJS module that exports a JSON file.
This config file enables you to execute test scripts. Open your terminal and enter this command:
npx wdio configThis opens the wdio config wizard.
Provide the necessary information as the wizard sets up the configuration.
* What type of testing would you like to do? **E2E Testing - of Web or Mobile Applications**
* Where is your automation backend located? **In the cloud using Testingbot or LambdaTest or a different service**
* What is the host address of that cloud service? **Hit Enter to select the default**
* What is the port on which that service is running? (80) **Hit Enter to select the default**
* Which framework do you want to use? **Mocha**
* Do you want to use a compiler? **No**
* Do you want WebdriverIO to autogenerate some test files? **Y**
* Where should these files be located? **Hit Enter to select the default**
* Do you want to use page objects (<https://martinfowler.com/bliki/PageObject.html>)? (Y/n) **Y**
* Where are your page objects located? **Hit Enter to select the default**
* Which reporter do you want to use? **Spec**
* Do you want to add a plugin to your test setup? **wait-for**
* Do you want to add a service to your test setup? **chromedriver, lambdatest**
* What is the base url? [https://ecommerce-playground.lambdatest.io/](https://ecommerce-playground.lambdatest.io/)
* Do you want me to run `npm install`? **Y**
-
Upon successful installation, a success message will be displayed.
It is time to run the actual test. Run this command in your terminal:
npx wdio
This would launch the default test generated by the WDIO configuration. You should see the logs of your test on your terminal. Below is the report from the spec reporter.
"spec" Reporter:
------------------------------------------------------------------
[chrome 114.0.5735.199 windows #0-0] Running: chrome (v114.0.5735.199) on windows
[chrome 114.0.5735.199 windows #0-0] Session ID: 046bebf8eb888dc2fb32e7b5c27c0710
[chrome 114.0.5735.199 windows #0-0]
[chrome 114.0.5735.199 windows #0-0] » \test\specs\test.e2e.js
[chrome 114.0.5735.199 windows #0-0] My Login application
[chrome 114.0.5735.199 windows #0-0] ✓ should login
with valid credentials
[chrome 114.0.5735.199 windows #0-0]
[chrome 114.0.5735.199 windows #0-0] 1 passing (7.6s)
Spec Files: 1 passed, 1 total (100% completed) in 00:00:23
00:00:23
ing down spawned worker
2023-07-18T17:07:08.985Z INFO @wdio/local-runner: Shuttng for 0 to shut down gracefullying down spawned worker ing down
2023-07-18T17:07:09.241Z INFO @wdio/local-runner: Waiting for 0 to shut down gracefully
2023-07-18T17:07:09.245Z INFO @wdio/local-runner: shutting down
Browsers Supported by WebdriverIO
All main browsers, including Chrome, Firefox, Safari, Microsoft Edge, and Internet Explorer, are supported by WebdriverI O. The Appium driver also supports iOS and Android devices.
Top Features of WebdriverIO
WebdriverIO runs on top of the Webdriver protocol. This protocol also powers other automation tools such as Selenium, and Protractor. By using this protocol, commands are sent from the test script to a driver that is specific to the browser being tested.
This offers some benefits among which are:
WebDriver BiDi-compatibility
WebDriver framework allows the testing of apps that use bi-directional text, which is frequently used in languages like Arabic, Hebrew, and Persian. This is referred to as WebDriver BiDi-compatibility.
Since BiDi text reads from the right to the left rather than the typical left-to-right form used in English, French, and other languages, it is distinctive. A highly specialized test automation framework that can manage the challenges of handling and interacting with this sort of text is needed for testing applications that use BiDi text.
WebDriver's BiDi compatibility allows automation testers to write scripts that interact with text in both left-to-right and right-to-left formats. It provides a full range of testing capabilities for BiDi text, enabling automation testers to write test cases for complete automation of end-to-end testing of applications written in bidirectional languages.
Supports Multiple Testing Frameworks
Web driver is highly feature-rich, it provides support for other test runner frameworks such as Mocha, Jasmine, and Cucumber, can be connected with WebdriverIO. The availability of adaptable test-runner interfaces enables this integration. You can integrate with different testing frameworks and libraries via test-runner interfaces.
Page Object Model (POM)
WebDriver uses the Page Object Model architectural pattern to build test automation frameworks for web applications. It is a mechanism for classifying a web application's pages, together with their properties and actions, into objects. In the POM framework, each page of a web application is represented as a distinct class.
The key benefit of adopting POM is that it makes scripts more customizable, reusable, and easy to maintain. In a separate class that can be quickly updated if the page changes, POM enables you to define the attributes and methods of web page objects. Selenium WebDriver uses POM, which makes it easier for testers to read and understand the code because it is structured more logically and consistently. This helps to write cleaner and more effective test code by reducing the amount of duplication that is frequently associated with writing test scripts. The Selenium WebDriver POM integration is an excellent solution to guarantee scalable and maintainable automation frameworks.
Smart Wait
One of the core concepts of WebdriverIO is auto-waiting. This essentially means waiting for elements to be available on the page before interacting with them. This feature was introduced in WebdriverIO v.6.
When elements are not available and you start interacting with them, it causes flaky behaviors. The Webdriver protocol offers two ways to wait on an element to show up:
- Implicit vs. Explicit: This allows you to specify how long the driver should wait for an element to appear. The default value is '0'.
- Explicit: Explicit waiting is a feature built into the WebdriverIO framework. You can use commands like 'waitForExist' to accomplish this. The framework polls for the element using this technique by issuing multiple 'findElements' commands until the timeout is reached.
Detailed Error Reporting
Detailed error reporting is at the heart of automation testing. It is inevitable, writes Aparna Gopalakrishnan in her post "How To Generate HTML Reports With WebdriverIO?".
Imagine having carried out automation tests without good reports. It would be difficult to find the bugs through automation tests. This would leave you with no actionable decision.
Fortunately, WebdriverIO has several third-party packages that enable comprehensive reports at the end of an automated test execution.
These reporters are classified into:
-
HTML reporters:
-
Non-HTML reporters:
Parallel Testing
Parallel testing entails executing multiple tests at the same time, concurrently. This will help you obtain feedback much more quickly, which is critical when working with large and complex software applications.
WebdriverIO as an automation test framework enables you to carry out parallel tests. Learn more about parallel testing on the official WebdriverIO documentation.
Extendable
In addition to what WebdriverIO provides, WebdriverIO has an extendable interface that allows you to integrate other services and reporters built by the community. Among these services is the LambdaTest Service - a WebdriverIO service that manages tunnel and job metadata for LambdaTest users.
Concise Test Syntax
WebdriverIO provides a concise way of writing tests. Instead of writing test like this:
driver.findElement(By.id("driver")).click();
You can also write it like this:
$("#driver").click();
Tabular Comparison Between Playwright and WebdriverIO
In this table, I provided some feature comparisons between WebdriverIO and Playwright
| Criteria | WebdriverIO | Playwright |
|---|---|---|
| Performance | Performant | According to astudy, Playwright is comparatively faster |
| Documentation | Developer friendlydocumentation | Comprehensive developer friendlydocumentation |
| Scalability | Highly scalable automation framework that is designed to cater to the needs of test automation. | Known for its scalability and can handle a growing number of tests. |
| Language and Syntax | JavaScript | JavaScript |
| Purpose and Scope | Browser Automation, Unit Testing, Functional Testing | End-to-End test |
| License | MIT License | Apache 2.0 License |
| Community Support | It is fairly new and does not have extensive support from the community | Playwright is also relatively new and came into existence in 2020. Its community support is growing and improving. |
| Custom Command | Allows the tester to add custom commands in the test script by calling the addCommand function |
Custom commands in Playwright refer to the creation of customized functions that can be used repeatedly throughout a script. Playwright supports custom commands. Playwright supportscustom helpers. |
| When to use | While testing web applications and native mobile applications for iOS-enabled devices. | While carrying out functional testing, end-to-end, and API testing. |
| Architecture | WebdriverIO's architecture is layered and built on top of NodeJS, which implements the JSON Wire Protocol. | Communication of Playwright is made over a single web socket. |
| CI/CD Integrations | Jenkins, Docker, and Bamboo | GitHub Actions, Docker, Azure Pipelines, CircleCI, Jenkins, Gitlab CI, Bitbucket Pipelines |
Conclusion
The choice of the right tool is essential to succeed in web automation projects. WebdriverIO and Playwright are two popular options that offer powerful capabilities and gained significant traction in the testing community.
WebdriverIO is a widely used test automation framework and offers a robust and mature solution for web automation. It supports multiple programming languages, and browser compatibility, and integrates well with other tools. Playwright, a newer entrant in the web automation space, offers multi-browser support, powerful automation, and a user-friendly API. Playwright's focus on reliability, speed, and behavior across browsers makes it appealing.
Choosing between WebdriverIO and Playwright depends on specific requirements, project complexity, and team preferences. Web automation provides several benefits, including faster releases, early bug detection, and better test coverage, leading to reliable and robust applications. In the rapidly evolving world of web automation, staying updated with the latest tools and technologies is crucial to driving the software development process forward.
Found this content useful? Follow me on Twitter (X) @eunit99
Top comments (0)