Web UI Automation Must Haves

There has alway been a need to have a better feedback loop a the desktop application or a web application under test. Many times we've had to ask [occasionally beg] a developer to build in some needed feedback functionality. But now that isn't so much the case when it comes to testing web applications. With the accessibility of Webdriver there are easier opportunities to get direct feedback from the web browser itself.

This direct ability to interact gives those writing automated tests the ability to answer some of the some of the most troublesome and nagging questions we all have needed the answer to.

1. Has the page fully loaded?
2. Have any JavaScript errors been thrown?

---

FYI: The examples below are from my WebdriverIO library, but the concept can be adapted to any variant of automated testing using Selenium or another automation framework.

---

1. Has The Page Fully Loaded?

There are many ways to resolve trouble maker #1. Many time's I've seen and done it myself, write my automation to wait for a particular element in the page to load or exist. This can be useful, but it does not scale.

The first part of the solution is to have the browser tell us its current document loading status. The is accomplished by hooking into the Document.readyState property of the browser. The Document.readyState property returns one of three responses.

1. loading
2. interactive
3. complete

This functionality of the browser document can be queried while the page is loading like the example below that returning TRUE if the document.readyState equals 'complete'. To access to the document property of the browser you have to use WebdriverIO/Selenium's ability to directly execute a JavaScript call.

export default class utilsLib {
    constructor() {}

    checkIfPageLoaded() {
        // console.log( `Done loading? ${browser.executeScript('return document.readyState', []) }` );
        return browser.executeScript('return document.readyState', []) === 'complete';
    }
...

Now that we have the ability to query the document.readyState, let's write a custom wait function called waitForPageToLoad to check if the browser has completed loading. But making sure to timeout if the waiting time is excessive.

...

    waitForPageToLoad() {
        browser.waitUntil(() => {
            // console.log('Waiting for the page to load');
            return this.checkIfPageLoaded() == true;
        }, 15000, 'Page never completed loading');
    }

...

2. Have any JavaScript errors been thrown?

After checking that a page has loaded, we now need to face the silent killer [no I'm not talking about heart disease]. The silent killers to any web application are JavaScript errors. There are times when a JavaScript error is thrown but it doesn't always cause a web application to fail a running test. A JavaScript error could cause a failure on functionality that isn't being tested at the moment or has automated tests for it. In any case, its good practice to always check for JavaScript errors after a web page is loaded.

To access the logging of JavaScript errors in the browser, you have to leverage WebdriverIO/Selenium's access to the JsonWireProtocol for Logging. There are five Log Types that can be accessed, but we are focused on browser so we can capture & parse the logs for potential JavaScript errors.

Log Type	Description
client	Logs from the client.
driver	Logs from the webdriver.
browser	Logs from the browser.
server	Logs from the server.

Along with Log Types, there are also Log Levels to consider [which any good API would provide]. To get down to the nitty gritty we are going to parse for SEVERE errors and just console log any other errors so they can be generally captured.

Log Level	Description
ALL	All log messages. Used for fetching of logs and configuration of logging.
DEBUG	Messages for debugging.
INFO	Messages with user information.
WARNING	Messages corresponding to non-critical problems.
SEVERE	Messages corresponding to critical errors.
OFF	No log messages. Used for configuration of logging.

As you review the checkForJavaScriptErrors function below, you'll see we're leveraging waitForPageToLoad to make sure the page is loaded before checking for JavaScript Errors. This allows for chaining of multiple levels of validation even before an assertion point is even executed.

...
    checkForJavaScriptErrors() {
        this.waitForPageToLoad();
        var logs = browser.getLogs('browser');
        logs.forEach(function (log) {
            if (log.level.toLowerCase() == 'severe') {
                if (log.source.toLowerCase() == 'javascript') {
                    console.error(`${log.source.toUpperCase()} ERROR: ${log.message}`);
                    expect.fail(`${log.source.toUpperCase()} ERROR: ${log.message}`);
                }
                else {
                    console.log(`${log.source.toUpperCase()} ERROR: ${log.message}`);
                }
            }
        });
    }


...

Leveraging the functions

As an example of how I've used by three super helper functions, I created a custom openUrl function as part of my class that will wait for the page to load and check for JavaScript errors.

...

    openUrl(path) {
        browser.url(path);
        this.checkForJavaScriptErrors();
    }
}

If you have any questions about WebdriverIO or Selenium based testing, please don't hesitate to leave a comment or message me directly.

Happy Testing!!!

---

Full Example Source

export default class utilsLib {
    constructor() {}

    checkIfPageLoaded() {
        // console.log( `Done loading? ${browser.executeScript('return document.readyState', []) }` );
        return browser.executeScript('return document.readyState', []) === 'complete';
    }

    waitForPageToLoad() {
        browser.waitUntil(() => {
            // console.log('Waiting for the page to load');
            return this.checkIfPageLoaded() == true;
        }, 15000, 'Page never completed loading');
    }

    checkForJavaScriptErrors() {
        this.waitForPageToLoad();
        var logs = browser.getLogs('browser');
        logs.forEach(function (log) {
            if (log.level.toLowerCase() == 'severe') {
                if (log.source.toLowerCase() == 'javascript') {
                    console.error(`${log.source.toUpperCase()} ERROR: ${log.message}`);
                    expect.fail(`${log.source.toUpperCase()} ERROR: ${log.message}`);
                }
                else {
                    console.log(`${log.source.toUpperCase()} ERROR: ${log.message}`);
                }
            }
        });
    }

    openUrl(path) {
        browser.url(path);
        this.checkForJavaScriptErrors();
    }
}