DEV Community: endtest

How To Record Automated Tests For Mobile Apps

endtest — Wed, 18 Mar 2020 13:48:00 +0000

Endtest allows you to create, manage and execute Automated Tests, without having to write any code.

You can use the open-source Endtest Chrome Extension to record a test for your website directly in the browser.

But did you know that you can also record tests for native and hybrid Android and iOS applications? 🤗

And you do that direcly from your browser.

What? 😱

That's right.

The Mobile Recorder allows you to record tests for native and hybrid mobile apps, directly in your browser.

What are the steps?

You upload the APK or IPA file.
The app is installed on a real mobile device.
The information for each element is extracted.
An interactive preview is shown in real-time in your browser window.
You record the test.
You save the test.

But how does it locate the elements?

It's no secret that our engine for Mobile Tests is using Appium.

We just fetch the locators for the elements that you interact with.

If we would rely on image recognition, that would lead to brittle tests.

We fetch the following locators:
• ID
• Accessibility ID
• Name
• Class Name
• XPath
• Coordinates

If you're not using Coordinates, you can easily execute the same test on different devices.

Coordinates are only used in extreme cases, when no other unique locator is available.

You can find more details in our Documentation.

Building The Most Powerful Test Recorder On The Market

endtest — Fri, 21 Feb 2020 15:09:14 +0000

Introduction

Endtest allows you to create, manage and execute Automated Tests, without having to write any code.

You can use the Endtest Chrome Extension to record a test directly in the browser.

And yes, it's open-source and free to use. 🤗

This extension works by recording all the steps that a user performs in the browser and converts those steps into an automated test.

1. How we record the steps

We record the following actions:
• Click
• Right Click
• Write Text
• Select option from dropdown
• Press Key
• Hover
• Scroll
• Open tab
• Close tab
• Switch to next tab
• Switch to previous tab
• Go back
• Go forward
• Access URL from navigation bar
• Entering an iframe
• Exiting an iframe
... and many more

Since we're talking about a Chrome extension, the only way to detect such events is through JavaScript and the Chrome API.

We used different JavaScript event listeners such as onclick, onchange, onkeyup, onmousemove, etc to figure out what the user does.

It's important to distinguish between the different actions.

For example, when a user is typing inside an input, you need to register that as a "Write Text" action.

But when the user pressed the Enter key or the Escape key, you need to register that as a "Press Key" action.

The user has the option to enable or disable the detection of Hover and Scroll events, since they're not relevant in all situations.

Even if a user enables the detection of Hover events, they will only be logged if the duration of Hover is above a certain number of miliseconds.

This is done to avoid recording unnecessary Hover events when the user is simply moving the pointer.

The Chrome API is used to detect actions which cannot be detected with JavaScript, such as when the user is opening a new tab or switching focus to a different tab.

2. Getting the right permissions

Being able to record everything that a user does in the browser might sound like a job for the NSA.

Luckily, Chrome is smart enough to let you declare what permissions you need in the manifest.json file.

They also review those permissions when you publish the extension in the Chrome Web Store.

And the user is informed about those permissions when installing the extension.

Here is what the permissions for our extension like:

  "permissions": [
    "activeTab", "<all_urls>","cookies","storage","tabs","webNavigation"
  ],

3. Locating the elements

It's no secret that our engine for Web Tests is using Selenium.

That's why we need to fetch locators for the elements that the users interacts with.

If we would rely on coordinates or image recognition, that would lead to brittle tests.

We fetch the following locators:
• ID
• Name
• CSS Selector
• XPath

Fetching the ID for an element might seem as simple as this:

window.onclick = function(e) {
   the_id = e.target.id;
}

But there are plenty of tricky situations:

The element does not have an ID
Multiple elements share the same ID
The ID is generated dynamically and changes with each refresh

This means that we always check for uniqueness and we try to detect and avoid using IDs which are dynamic.

The same rules apply for the Name attribute.

CSS Selectors can be generated by using the using the attributes from an element and the parent nodes.

Let's take a look at this example:

<html>
   <body>
      <div class="login-form">
         <input id="mat-input-6" type="text" placeholder="Email">
         <input id="mat-input-9" type="password" placeholder="Password">
      </div>
   </body>
</html>

We can easily spot that the IDs are dynamic, so we have to avoid them.

The basic CSS Selector for the Email input would look like this:

body > div > input[type=text]:nth-child(1)

But we can easily construct something more reliable:

input[placeholder="Email"]

Most people ditch XPaths for CSS Selectors, thinking that they're just as powerful, but slower. Big mistake.

XPaths have some hidden tricks up their sleeves.

The XPath equivalent to the previous CSS Selector would look like this:

//input[@placeholder = "Email"]

But here's a situation where XPath can save your life:

<html>
   <body>
      <ul class="items">
         <li class="item">Apple</li>
         <li class="item">Orange</li>
         <li class="item">Banana</li>
      </ul>
   </body>
</html>

Let's try to fetch a locator for the Orange list item.

CSS Selectors do not have the ability to locate an element based on the text, but XPaths do.

The XPath for the Orange list item is:

//li[contains(text(),'Orange')]

We even offer advanced settings for detecting locators:

Using classes in CSS Selectors and XPaths can also be tricky, since they can only appear in the element when you click on it.

Let's take a look at this example:

<html>
   <body>
      <div class="container">
         <input type="button" class="login-form">
      </div>
   </body>
</html>

We want to locate the input type="button" element.

But when we click on that input, it gets 2 extra classes:

<html>
   <body>
      <div class="container">
         <input type="button" class="login-form active focus">
      </div>
   </body>
</html>

Since we're recording the event at the moment of the click, our Chrome Extension would record the element as having 2 extra classes.

If this would happen, our engine would throw an "Element not found" error when the test would be executed, since those 2 extra classes are not appended if the element isn't clicked.

That's why allow the users to have a list of classes that our Chrome Extension can simply ignore:

4. Going the extra mile

We always make sure our users get the best experience.

Steps for taking screenshots and adding assertions can be added directly from the recorder.

The recording process can also be paused and resumed.

You can find more details in our Documentation

Q&A

1. Why do we also offer a cross-browser cloud infrastructure?

We take testing very seriously.

That's why our cloud platform offers real browsers on Windows and macOS machines and mobile devices.

We encourage developers to avoid using headless browsers on Linux machines for their automated tests, since those browsers do not have the same behavior as the real browsers used by users.

2. Why record the tests with JavaScript and execute them with Selenium?

Since our recorder had to be a Chrome extension, we had no other choice than to use JavaScript.

Our engine for Web Tests is using Selenium, because it's open-source, constantly improved, mimics real user behavior and it works with all the major browsers.

We advise developers to avoid using JavaScript-based libraries for executing automated tests in the UI, since those libraries have severe limitations and do not mimic real user behavior.

For example, a JavaScript-based library for testing the UI can easily click on an element which is completely covered by another element.
A real user and a Selenium test cannot do that.

A JavaScript library for testing the UI cannot perform file uploads, cannot manage iframes and cannot manage multiple browser tabs.

Currently, there isn't any JavaScript library for UI testing that works on all the major browsers and devices.

How Endtest Is Using Machine Learning To Make Your Tests More Stable

endtest — Mon, 13 May 2019 15:21:22 +0000

It's no secret that we use Machine Learning here at Endtest.

And it's not just a buzzword that we throw around to attract users.

In this article, we'll show you how it works.

Endtest is a cloud platform where anyone can create, manage and run automated tests for Web and Mobile Apps, without having to write any code.

We're talking about functional tests, which interact with the app in the same way that a user would, through the UI.

Our engine is using Python and Selenium to interact with the elements from Web Applications and Python and Appium to interact with the elements from Mobile Applications.

In order to find an element with Selenium and Appium, you need to provide a locator for that element.

Let's take a look at a quick example.

The Email address input from the Sign Up form at Airbnb:

When creating a Selenium test, you could use one of the following locators to find that input:
ID = email-signup-email
Name = user[email]
Class Name = _14fdu48d
XPath = //*[@id ="email-signup-email"]
CSS Selector = #email-signup-email

You could go the extra mile and write your own custom XPaths based on any attributes which uniquely identify that element:

XPath = //*[@type="email"]
XPath = //*[@autocomplete="username"]
XPath = //*[@placeholder="Email address"]
XPath = //*[@type="email" and @autocomplete="username"]

When you write such a test, you would only use one of those locators.

Now, I want you to choose the locator which you think is the most reliable.

Alright, did you pick one? Great, we're going to need it for the next step.

Let's imagine that Airbnb would go through a revamping.

And that input would now look like this:

What locator did you pick?

You picked ID, didn't you?

Sadly, your test will now fail and it will give you a false negative.

Seems you need to update it.

Someone working on tests always goes through that, the pain of maintenance.

Updating one step isn't time-consuming, but it's rarely just one.

That's one of the many problems that we're solving with our platform.

Just enable the Machine Learning option from the Advanced Settings:

And when you run your test, our engine will remember between 50 and 100 ways to locate your element.

It will remember all the attributes, XPaths and CSS Selectors. details about the neighbours and the element's role in the workflow.

All that information will be useful when changes in your application will occur.

If those changes break your tests, our engine can easily update your tests to match the latest changes.

All you have to do is enable the Self-Healing option when you run your test:

When that test will reach a step that no longer corresponds to the new UI and workflow, it will fetch all the information that it gathered about the element, it will scrape the page in multiple threads, it will perform a ton of calculations and rankings to determine if the element is still on that page and determine what are the new locators.

It will update your test in order to match the latest changes from your app.

From our thorough experiments, we got an accuracy of 98%.

Our algorithms can actually differentiate an intended change from a bug.

Each step that was updated by our engine will be detailed in the Results section.

In the Machine Learning world, this is known as Supervised Learning.

10 Unexpected Ways To Improve Your Automated Tests

endtest — Wed, 18 Jul 2018 11:59:21 +0000

Should I use Selenium or something else? Should I make a list of everything that needs to be automated? Should I ask for help?

Sounds familiar?

I know the feeling. I felt the same way.

Whether you are a beginner or an expert, chances are some of these points will help you bring your automated testing skills to the next level.

Here are 10 unexpected ways to help you improve your automated tests.

1. Embrace the ever-changing technology trends

Coding your own automated testing framework can make your life a living hell. So, how about you avoid perdition and accept innovation?

The worst part is that you're not going to realize that…at least until it's too late.

Creating a basic test suite is a piece of cake, but would you enjoy maintaining the entire codebase when 478 out of your 626 tests will be failing due to different errors, right before the big release?

…

That's right.

Yep, your entire team will end up doing manual testing for the next couple of days. Bummer, huh?

If you're thinking "That's totally not me", ask yourself if you have the time to handle:
• Creating a stable cross-browser cloud infrastructure for your tests.
• Implementing image comparison algorithms for visual checks.
• Implementing video recording for your test runs.
• Implementing a schedule for your tests to run on a daily basis.
• Integrating your tests with your CI/CD system.

If the answer is "yes", you can stop reading from here on.

If the answer is "no", the next question should be "alright, what's the alternative?".
Good news is that companies have started migrating to cloud platforms that provide testing infrastructure (IaaS, PaaS and SaaS).

Here's what I could find about them, after doing some extensive research:

As for Cucumber or Behat, they are just libraries so I couldn't find a spot for them here.

2. Make stability a priority.

If your test passed 99 times and failed once, the bottom line is that your automated tests are unstable, as unpleasant as acknowledging that may be.

Having 2 stable tests instead of 5 unstable ones is always better. Those unstable tests will just test your patience and force you to manually check the functionality over and over again.

You already know that it's not a smart move to move on to the next test case until the one you're working on is completely stable.

That sneaky "I'm going to come back and fix it anyway" lingering in your mind won't happen and you're just going to end up doing extra work to fix it.

3. It's OKAY to be OBSESSED.

Don't forget to focus on the negative testing scenarios, that's where the bugs crawl, they don't like the clean happy path.

Create a test for each bug you find while manually testing, so you know it will never cheap-shot you.

4. Visuals are important.

Even if your tests are interacting with the applications via the UI and you're checking for the existence of some elements, that doesn't mean everything will be pixel perfect.

That's why it's important to add screenshot comparison steps that visually check elements, pixel by pixel, against already existing screenshots.

I wrote a short technical article - - that you might find useful -- about doing that, a few months ago.

5. You. Can't. Automate. Everything.

Sadly, there are some things that can't be automated in a stable and efficient manner. The sooner we accept that, the better.
Take reCAPTCHA for example, you're probably going to find yourself in need to disable it for the requests coming from your test machines.

Dwelling on automating the impossible will just waste your time.
Luckily, there are smart workarounds out there. All you have to do is find them.

6. Cross Browser: You DO need it.

"Well, our web application is working great in Chrome…"
"And that's what most people are using anyway…"
"Hmm, it would take us some time to make the tests run on all the browsers…"
"Some users have been reporting some bugs, maybe they're using some old version of Chrome or … you know, users are just stupid sometimes."

Curiosity drives you to go around and ask for statistics about what browsers people are using to access your web application.

That's how you get this lovely pie chart:

How f****d are you?

This is when you know you need to go Cross Browser, there is really no other way to go.

It may be complicated and, sometimes, expensive, but it's always worth it.

And if you're going to go, go all the way:
• Don't rely on headless browsers.
• Use Windows machines for testing in Chrome, Firefox and IE.
• Use Windows 10 machines for testing in Edge.
• Use Mac OS machines for testing in Safari, Chrome and Firefox.

If you're going to go codeless, the only platform that offers the above - while being…well, codeless - at the moment, is Endtest.

7. Remember the little things.

Sometimes, I tend to forget those little things that do matter.
Here are some questions to ask yourself when you think you're done:

• When was the last time you tested the META tags from the Page Source?
• When was the last time you tested the cookies?

These things are surprisingly important for ranking, marketing and tracking purposes.

8. Sometimes, Mobile comes First.

Those who are working on B2C products already know that most users access their product through their mobile device.
Those who are working on B2B products need to start considering the same thing.

9. Not just the testers, but everyone should write tests.

Since the Product Owner and the Business Analyst are the ones who know best how the software and flows should work, they're the best people to take part in the quest that test-writing can be.

Their take will always be useful when writing the tests for happy paths.

This experience is a win-win, since it also helps them gain a little more insight on how their User Stories were implemented.

But they might not know how to write code….

For this kind of scenario, going codeless is the only way to go if you're not willing to spend months on teaching them how to do it. If you are, I must say I admire your patience.

10. Don't be afraid to ask for a little bit of help.

Since the entire company will benefit from those automated tests, everyone will gladly pitch in if you need a little hand ( even if you know better! ).