DEV Community 👩‍💻👨‍💻

DEV Community 👩‍💻👨‍💻 is a community of 966,155 amazing developers

We're a place where coders share, stay up-to-date and grow their careers.

Create account Log in
Pablo Calvo
Pablo Calvo

Posted on

WebDriver vs Chrome DevTools (Speed Test)

Today I got inspired by @FedericoToledo to write a test script that can automate a simple clicking game, but with a focus on executing as fast as possible.

This challenge inspired me to use my favorite Web Testing Framework webdriverIO which support devtools and webdriver protocols out of the box with minor setup. So I decided to try and see which protocol executes faster.

Let's explain the details first.

In order to make your tests scripts execute actions against a website your test framework has to use an interface which is actually capable of controlling the browser.

There are 2 very popular interfaces that allow this to happen. Selenium uses WebDriver (w3c protocol) which is today the most popular one, but there is also Chrome DevTools which is native to chromium browsers and has some benefits and disadvantages over its competitor. In this post we are focusing on speed, and spoiler alert our winner is devTools.

Here is why!

WebDriver sits in the middle between the test scripts and the browser, and uses http traffic to communicate back and forth which can cause a small latency between the execution and the response from the browser.

Alt Text

On the other hand Chrome DevTools allows direct control over the browser by using the Puppeteer library which is finally used by WebDriverio. This means direct execution and less latency on the results.

Alt Text

Let's do the test

The idea of the game is to click the numbers from 1 to 50 as fast as possible, but only the next 25 are displayed. So if you click 1 then the 26 appear. Game

I created a simple script using WebdriverIO on standalone mode so that it runs fast without all the test framework setup.

const { remote } = require('webdriverio');
(async () => {
    const browser = await remote({
        logLevel: 'trace', capabilities: { browserName: 'chrome'}
    })
    await browser.url('https://zzzscore.com/1to50/en/?ts=1592666149743') // navigate
    for(i = 1; i<51; i++){
        var element = await browser.$('//div[text()="@id"]'.replace('@id',i))
        await element.click(); //click each number
    }
    var result = await browser.$('.level');
    console.log(`The result is ${await result.getText()}`);
    // await browser.deleteSession()
})().catch((e) => console.error(e))

By default webdriverIO v6 will try to connect using devtools protocol unless a explicit WebDriver path is specified.

To run the script just type

node <scriptname>.js

DevTools Result

The first thing you will notice when executing the script is this line of log: INFO webdriverio: Initiate new session using the devtools protocol which indicates the used protocol.

And you get this result: The result is 0.85. Which is really fast, it meant at least 50 browser interactions in less than a second.

WebDriver Result

In order to execute the same script on WebDriver protocol, we need to specify the hostname and path when creating the remote object. We also require a running webdriver server. I recommend installing the webdriver-manager package which simplifies the process a lot.

Once the webdriver is running (by default it runs on localhost:444/wd/hub), then just update the code.

...
 const browser = await remote({
        logLevel: 'trace', capabilities: { browserName: 'chrome'},
        hostname: 'localhost', path:'/wd/hub'
    })
...

Again you will notice that the webdriver protocol is specified: INFO webdriverio: Initiate new session using the webdriver protocol.

And the result is: The result is 3.801 which is impressing but it is 4 times slower than the first execution and places webdriver in the second place.

Final Thoughts

We can easily conclude that devtools protocol execute scripts way faster which makes me consider it as a scripting and scraping tool. But there is no free lunch, webdriver is now days the most used interface and the one that has support for cross browser testing, devices and even desktop apps.

I will also mention the great capability of webdriverIO to support both protocols out of the box. It really matters a lot when a tool allows you the power of choice.

Credits for the protocols images to webdriverio website

Top comments (6)

Collapse
 
klamping profile image
Kevin Lamping

This is a neat little exercise! I keep trying to think of ways to make it faster, but haven't had any success. I was able to switch to using a CSS-based selector targeting off the 'z-index', but it didn't seem to improve speeds at all. I've heard that xpath is slower than CSS, so I was hopefuly, but seems like it didn't matter that much in this circumstance.

Collapse
 
pjcalvo profile image
Pablo Calvo Author

Hi @kevin , thanks for commenting out. Yeah I have also heard that xpath strategy is slower than css. But so far it is like a myth, everybody believes it but nobody has seen it.

I think it actually depends on the complexity of the website DOM, but mostly on the xpath strategy: complicated locators with text validations sounds like they might decrease performance. (Not sure until I do an experiment).

Also, I was able to successfully beat the score on this post by injecting the javascript which locates (using javascript xpath locate function) and dispatches the click event using webdriverIO. At some point the execution was faster than the actual website response, so I don't think there can be a faster response.

Check out the code. gist.github.com/pjcalvo/6875f07ec6...

And the score dev-to-uploads.s3.amazonaws.com/i/...

Collapse
 
klamping profile image
Kevin Lamping

You read my mind on using browser.execute to trigger the tap event. I was trying that last night, but couldn't get it to trigger the event on the right element. Nice work figuring it out!

And yeah, I don't think that score is going to be beaten, aside from completely cheating and just bypassing the clicking all together and somehow triggering the "completion" function.

Collapse
 
whimboo profile image
Henrik Skupin • Edited on

Thanks for this post and the results. As one of the developers working on geckodriver I had a special interest to see why it is actually slower. And to help others please let me add some more details why using webdriver is slower...

The underlying issue manifests in the WebDriver specification for Element Click. When you read through that chapter you will notice that in step 10 additional steps are taken beside the simple click, which have to check if a page navigation has actually been started. It takes some milliseconds and as such delays the response of the command. Exactly that is what makes webdriver based implementations slower.

That part of the spec has been added in the past to keep in sync with the pre-WebDriver behavior that helped users of Selenium to more easily create their tests. Nowadays it might make sense to lift this up a bit, and maybe require an opt-in for this behavior. As such I created a webdriver specification issue for discussion at github.com/w3c/webdriver/issues/1537.

To circumvent that behavior you could set the Page Load Strategy to None, which means that there won't be such a check in Element click. As result you will get similar timings. In my case I get ~0.9s. But the drawback is that each of the navigation commands (Navigate To, Refresh, Back, Forward) are also affected and would need custom code in the test to ensure the page has been loaded.

Collapse
 
pjcalvo profile image
Pablo Calvo Author

I just read this. Awesome reply will dig deeper

Collapse
 
whimboo profile image
Henrik Skupin

And what I actually missed when writing my reply, the Perform Actions command does not rely on this behavior. That means there won't be a need to change the page load strategy, but benefit from the performance win out of the box.

DEV has this feature:

Settings

Go to your customization settings to nudge your home feed to show content more relevant to your developer experience level. 🛠