As a test engineer or a developer, you want to be able to trust that your automated UI tests for Lightning UI are accurate and fail only if there really is a regression. If you are using Selenium then this blog post will most certainly bring you a large portion closer to reaching this goal.
Not familiar with Selenium WebDriver? It is the de-facto standard open-source tool for testing web applications like the Salesforce app. Learn more about Selenium WebDriver on its homepage or this Wikipedia article. This 5 Minute Getting Started Guide leads you through the first steps to run tests automatically.
The Problem
Test engineers responsible for UI Test Automation know only too well the situation when a test attempts to interact with a component on a page in the browser before the component has been fully loaded. An exception is thrown, resulting in a test failure. Such a “false positive” costs time, resources, and a certain steeling of nerves to investigate and fix. This is a major cost driver for UI test automation.
The Solution
Selenium WebDriver offers some powerful wait functions out of the box. Unfortunately not every test engineer or developer puts them to good use. Let’s see how we can use them effectively when testing the Lightning UI. We will also show that the widely used Sleep calls are only needed under certain circumstances and can safely be removed in all other cases.
Selenium WebDriver Wait functions
Selenium WebDriver provides three types of waits:
- Implicit Wait
- Explicit Wait – WebDriverWait
- Explicit Wait – Fluent Wait (Java only)
Selenium WebDriver can be used in a number of popular programming languages, including C#, Groovy, Java, Perl, PHP, Python, Ruby and Scala. Since FluentWait is specific to Java, the code samples in this article will focus on this language.
Implicit Wait
By setting implicit wait you are telling Selenium WebDriver how long to wait when searching for an element that is not immediately present on the page. Of course the driver does not “sit around” for the configured amount of seconds, but polls the page until the element has been found. If the element is not found within the specified time, the driver throws an exception.
You may ask “What is a good value to choose when testing Lightning?” Before I answer this, let’s first take a look at how implicit wait works.
Example Usage:
01 driver.manage().timeouts().implicitlyWait(3, TimeUnit.SECONDS);
02 driver.findElement(By.id("Login")).click();
03 driver.findElement(By.xpath("//input[@title='Search...']")).click();"
On line 1 in the above code, the WebDriver instance is configured to wait for up to 3 seconds for elements to appear. This timeout is valid until you set another value or the WebDriver instance has ended.
On line 2, WebDriver is looking for the Login button to appear on the web page. If the element is not located on the web page within that 3 second time frame, an exception is thrown. If the element appears within 3 seconds, e.g. after 2 seconds, the click() method is executed.
Now comes the crucial part to understanding the benefit of this wait type!
Let’s assume the click on the Login button in line 2 initiates the login into the Salesforce application. As you all know from experience, this may take a long time. However, the implicitWait value does not have to be so high that it will surely cover that! The reason for this is that the click call does not return right away to the test execution. In some surprisingly comprehensive fashion, the Selenium WebDriver will wait for this click action to complete. So when we reach line 3 in the example code, the whole page has already fully loaded and finding the Global Search field will succeed right away. No extra wait may be needed.
The code on line 3 shows the currently valid locator for the Global Search field. For more details on writing test automation involving this field, please see this blog post.
Granted, waiting for the click() action to complete was not working so well in Selenium’s past, but this tool has evolved a lot in the last couple of years; so I recommend to let it do its job and only interfere when necessary. When automating testing of the Lightning UI, go with a low value <= 5 secs.
If you have problems with automating a certain test step, then I highly recommend to not bump up implicit waiting. Instead, go with one of the next two waiting types as they are much better suited to fix your problems!
Explicit Wait – Webdriverwait
Using WebDriverWait allows you to define a wait for a certain condition to occur before proceeding further in the test, hence the term “explicit wait”.
It’s a typical scenario in Salesforce applications that data comes from third-party data sources, e.g. a legacy database. In such a case, the Salesforce application has no control over the responsiveness of the external system and so you see a text saying “Loading…” before the data appears in the browser.
Another common scenario is that a button is disabled until background activities have finished, and it becomes necessary to wait for the button to be enabled after the activities are complete.
Example Usage:
01 WebElement myDynamicElement = (new WebDriverWait(driver, 30))
02 .until(ExpectedConditions.presenceOfElementLocated(By.id("my3rdPartyElement")));
On line 1, a new WebDriverWait instance is created and configured to use the current WebDriver instance and a wait time of 30 seconds.
On line 2 we tell this WebDriverWait instance to wait for the presence of the my3rdPartyElement element and return it within 30 seconds. If nothing is found after that time, a TimeoutException is thrown.
Selenium WebDriver’s ExpectedConditions class provides many convenient methods that provide the ability to wait until an element meets a certain condition, such as becoming visible or clickable. For a full list of methods provided by this class, please refer to the Selenium WebDriver JavaDoc here.
If you need even more control over how often WebDriver should search for an element or if a certain exception is to be ignored when searching, then you need to use Fluent Wait, which I discuss next.
Explicit Wait – Fluent Wait (Java only)
The FluentWait class is available in Selenium WebDriver’s Java version only to get on par with the ease of use of WebDriverWait in other programming languages. FluentWait allows you to tell the WebDriver instance to not only wait for a specific condition to occur but to also check for the condition at a specified frequency, and to ignore specified exceptions while waiting. While the WebDriverWait class does not give you these options, everything else, including the use of ExpectedConditions, is identical in the FluentWait class.
Example Usage:
01 Wait<WebDriver> wait = new FluentWait<WebDriver>(driver)
02 .withTimeout(300, TimeUnit.SECONDS)
03 .pollingEvery(5, TimeUnit.SECONDS)
04 .ignoring(NoSuchElementException.class);
05 WebElement dynamicElementReference =
06 wait.until(ExpectedConditions.textToBe(By.id("my3rdPartyElement"), "Click me!"));
07 dynamicElementReference.click();
On lines 2 and 3, the FluentWait instance is configured to wait for up to 5 minutes and to check for the condition every 5 seconds.
On line 4 the instance is configured to ignore the NoSuchElementException exception. The importance of this can be seen on line 6 because the my3rdPartyElement element may not be present at the beginning of the wait. Only when the element is finally found does it make sense to check for the text. If the exception was not ignored, the wait would only work if the element were already present.
On line 7 the element finally gets clicked if the element appears within 5 minutes and has the expected text.
With the above examples, it is easy to understand that the two explicit wait types are sufficient to cover situations where implicit wait fails. However, sometimes these wait functions can be too elaborate for the situation and the judicious use of Thread.sleep() can be a simple and helpful alternative. Unfortunately, it is often overused.
When to Use Sleep Calls
All the languages which are supported by Selenium offer a call to suspend execution for a given period of time. To keep it simple, we will use Java in the code examples, but the given information is applicable to all of them.
Thread.SLEEP Overuse
Thread.sleep() is a core Java method, and while not Selenium WebDriver-specific, you can find many of those calls in typical test projects. The call does exactly as its name suggests: it suspends a test for a given amount of time.
A test code sequence like the below example takes about 5 seconds to run and works absolutely fine without any use of a Thread.sleep() call!
01 WebElement inputBox = wd.findElement(By.xpath("//input[@title='Search Salesforce']"));
02 inputBox.click();
03 inputBox.clear();
04 inputBox.sendKeys("CS88 Corp");
05 inputBox.click();
06 wd.findElement(By.xpath("//span[@title='CS88 Corp']"));
07 inputBox.sendKeys(Keys.RETURN);
08 wd.findElement(By.xpath("//a[@title='CS88 Corp']"));
Despite this, the code in many test projects looks like the following and will need 10 extra seconds to run:
// Overuse of Thread.sleep() - Don't do this!
01 WebElement inputBox = wd.findElement(By.xpath("//input[@title='Search Salesforce']"));
02 inputBox.click();
03 Thread.sleep(2000L); // Wait 2 seconds
04 inputBox.clear();
05 inputBox.sendKeys("CS88 Corp");
06 Thread.sleep(2000L); // Wait 2 seconds
07 inputBox.click();
08 Thread.sleep(2000L); // Wait 2 seconds
09 wd.findElement(By.xpath("//span[@title='CS88 Corp']"));
10 inputBox.sendKeys(Keys.RETURN);
11 Thread.sleep(2000L); // Wait 2 seconds
12 wd.findElement(By.xpath("//a[@title='CS88 Corp']"));
13 Thread.sleep(2000L); // Wait 2 seconds
Believe it or not, a sleep after the last step, as shown in line 13, is found more often than you think!
I’ve seen numerous tests that abuse the Thread.sleep() method by inserting these calls unnecessarily or increasing their value just in case there is a test failure at that step. Remember that time elapses no matter what. If you use the method to wait for 30 seconds just to cover a possible performance bottleneck, it will also wait 30 secs when there is no such bottleneck. And what happens if the bottleneck persists for 31 seconds? You will have a test failure on your hands!
WebDriverWait or FluentWait will work much better in such a situation. Additionally, the delays can quickly add up to minutes; and in larger test projects- hours of wasted time!
Proper use of Thread.sleep()
Should Thread.sleep() calls be totally banned from your test projects? Well, no. There are scenarios where you need only to bridge a brief moment in time, e.g. a second or two, in order for the Lightning UI to enter the desired state. In such a case, the use of a one or two second Thread.sleep() call is preferable to the use of an elaborate FluentWait or WebDriverWait call which spans a couple lines of code and is not easy to read or maintain.
Scenario 1 – the browser requires some time to react to JavaScript command
Under circumstances like those pointed out in this Knowledge article, it may be necessary to replace WebDriver’s click() method calls with a JavaScript call. The piece of code we are interested in is referred to by the article as “Code Sample 3” which — for your convenience — is shown here:
01 public void jsClick(By locator) {
02 jsClick(driver.findElement(locator));
03 }
04
05 public static void jsClick(WebElement elem) {
06 ((JavascriptExecutor) driver).executeScript(
07 "arguments[0].click();", elem
08 );
09 try { Thread.sleep(1000L); }
10 catch (InterruptedException ie) { ; /* ignore */ }
11 }
Why is there a one second sleep on line 9? Let me remind you of one sentence I wrote above: “In some surprisingly comprehensive fashion, the Selenium WebDriver will wait for this click action to complete.” But in this case, we are using a JavaScript call, Selenium has no knowledge of this fact and hence it cannot perform the task of waiting for the action to complete. By introducing this small delay of one second, the web page can react to the call and Selenium applies its usual waiting algorithms. We use this in automated tests and it performs with about 95% accuracy.
Scenario 2 – Selenium is faster than the browser’s renderer
Recently, I needed to introduce a small delay during the creation of a new Event record. In the process, the test selected an event type from a drop-down. The test executed so fast that there wasn’t sufficient time to render the option on the screen. In this situation, I had two choices. Either a) Insert four or five lines of code using FluentWait, allowing for the event type to be rendered while coping with any exception that WebDriver may encounter in the process; or b) Add one line of code to create 2 seconds of wait time using Thread.sleep(). I went with the latter because it avoided unnecessary complexity in an otherwise short and simple method.
Conclusion
As shown above, by introducing explicit wait statements and pruning unnecessary sleep calls from your test projects, you will be able to reap these numerous benefits:
- Much higher test pass rates without re-run; >95% can be achieved. This means that a test fails with a false positive only one time out of 20 runs!
- Test failures that are more likely to be an indication of a real issue.
- Increased ROI of test automation.
- Fewer resources wasted on triaging false positives.
- Reduced execution time allows for more tests to be run or less hardware to be occupied.
- Delays introduced due to events like waiting for response from a call to web-services or a third-party database are handled in an efficient, yet robust manner.
Please share this article with test engineers in your organization who may not (yet) be following the Salesforce Developer blog.
Oldest comments (0)