A step-by-step tutorial in codelab format for Android UI testing

Hi everyone! My name is Azamat Cherchesov and I am a senior Android developer at Kaspersky. We have been developing mobile applications for more than 10 years and know from our own experience that testing is an important stage in the entire process of creating new apps and preparing each new release. For Android applications Espresso is the default recommended framework, but it contains several known problems that do not allow us to cover all our needs for auto testing.

What do we want?

Readability

Espresso has some problems with readability. Its syntax is built on a hierarchical system of matchers, with a sufficiently high level of nesting, making the code hard to read and maintain.

@Test
fun espressoTest() {
  onView(withId(R.id.toFirstFeature))
    .check(ViewAssertions.matches(
     ViewMatchers.withEffectiveVisibility(
       ViewMatches.Visibility.VISIBLE)))
  onView(withId(R.id.toFirstFeature))
    .check(ViewAssertions.matches(ViewMatchers.isClickable()))
  onView(withId(R.id.toFirstFeature)).perform(ViewActions.click())
}

The above is the simplest example, but even this shows a readability problem.

For contrast, here’s an example of how this code could look differently:

@Test
fun readableTest() {
  MainScreen {
    firstFeatureButton {
      isVisible()
      isClickable()
      click()
    }
  }
}

We would also like to have a declarative approach when writing autotests, which is also not possible with Espresso.

Stability

Espresso does not work well with UI elements that are not displayed immediately. You can try idling resources, but it is difficult to implement. Espresso tests are not stable — if you run a simple test several times it can fail once or twice for some unpredictable reason (a common problem for most testing frameworks). You can read more about unstable clicks in Espresso and how we improved it on our website.

Detailed and easy-to-read logs and reports

After testing, there is no logged sequence of actions that were successfully completed. In case of a fail, it can be difficult to understand at what step in a particular test script something went wrong — Espresso simply provides a long trace with an error.

Executing ADB commands

Sometimes the default Android instrumental test tooling is not enough: a test case could imply that the device’s internet connection is off, brightness is set to a certain level, network is slow etc. Most of these conditions could be easily set beforehand on the CI side, but what if you want to turn off the internet during the test? Chances are you would need to fall back to manual testing. Here are some reasons why ADB may be needed in Android tests. In order to:

• perform standard operations like push/pull, manipulate files, install additional APKs, uninstall unwanted, etc.
• emulate anything using ADB emu: call simulation, fingerprint interaction, slow internet simulation, set a fake geolocation, etc.

Interaction with other applications and system elements and interfaces

You won’t even be able to interact with the system dialog using Espresso. Also you may need to perform some actions in another application during the test.

Google recommends using UiAutomator, but its syntax is also not very readable and it is also different from Espresso. In addition, we would like to see much more stability and speed of operation.

Screenshots

Mobile applications are often localized into multiple languages. Screenshot testing helps technical writers and localization engineers to better understand in what context new strings will be used and ensure they are suitable for the context and aren’t too long. It also helps with design testing — you can take real screens and compare them with mockups. We also want to see screenshots after running e2e tests (this can be useful when running tests using continuous integration to generate detailed reports).

You may know some screenshot frameworks, but in addition to the native approach to writing screenshot tests, we would like to be able to take screenshots of the application in different states, including negative scenarios (internet lost, server returned an error, record not found in the database, subscription or license expired). To do this, we need an easy way to transfer the application (or some screens) to the required state directly from the test screenshot code without unnecessary external manipulations.

Kaspresso comes to the rescue

As a result of much research and searching for best practices and solutions, the open source framework Kaspresso was created to cover all the needs for testing Android apps and has become a mature solution with a large team and an interested community. Take a look at some features Kaspresso can offer you:

• Built-in protection against flaky tests
• Page object pattern out of the box
• Declarative approach for writing tests
• Ability to interact with other applications and system elements and interfaces
• Human readability with Kotlin DSL wrappers over UiAutomator and Espresso
• Detailed logs and reports (logs, view hierarchy, screenshots, video etc.)
• ADB support
• Screenshot testing with native approach (with dark mode support)
• Compose support
• Allure support
• Robolectric support
• Easy migration from Espresso
• Flexible configuration options
• Automatic artifact pulling after test execution

You can find detailed information on our website in the Home and Wiki tabs.

A little dive into the details

Protection against flaky tests

Kaspresso provides interceptors and wrappers over Espresso and UiAutomator that catch exceptions in test code execution and retries with some additional logic.

In a nutshell, interceptors wrap calls to check and perform methods.

Beyond simple retries, Kaspresso intercepts, and deals with, cases when scroll is needed or some elements are not visible due to system dialogs or long content loading.

We recommend you try it now or learn more about it here.

Interaction with Android OS

Kaspresso provides the Facade class Device. With this class you can:

• push/pull files
• enable/disable network
• give permissions like a user
• emulate phone calls and fake geolocations
• take screenshots
• enable/disable GPS
• rotate device or press system buttons
• enable/disable accessibility
• change app language
• collect and parse logcat output

(see more about the Device class here).

Readability

First of all, let’s have a look at how beautiful, readable and concise Kaspresso tests are!

@Test
fun readableTest() {
  MainScreen {
    firstFeatureButton {
      isVisible()
      isClickable()
      click()
    }
  }
}

Under the hood, Kaspresso uses Kakao, — a Kotlin DSL wrapper over Espresso. From Kakao our framework inherited two main concepts.

The first is KView, a special representation of the interface element with which interaction will occur in the test. Using KView saves us from constantly calling the onView method; now we just need to put matchers in the KView constructor once.

object MainScreen: Screen<MainScreen> {
   val firstFeatureButton = KButton{ withId(R.id.toFirstFeature) }
}

The second concept is the Screen class (an implementation of the page object pattern), which describes all the elements that will be interacted with during the test. The concept comes from web development and is about creating a description of the screen that is visible to the user. This object does not contain any logic, which allows you to describe screens and their elements in a separate file and interact with them from the test class code. Thus, page objects are completely independent, thereby achieving maximum reusability.

Kotlin DSL over Espresso and UiAutomator provides a declarative approach for writing tests. Tests archives human readability and becomes easier for the next support and extensions.

Ability to execute ADB commands

Espresso, unlike Appium, does not allow you to execute ADB commands. However, with Kaspresso you have the opportunity not only to execute ADB commands in native tests, but to do it directly from the test code. The implementation is not that simple. As I go into more detail in this article, I will make it more complex. See this article for details of how this became possible.

Screenshots

Kaspresso is not the first or only framework that allows you to create screenshot tests. But unlike other tools, our framework provides several killer screenshot features.

Firstly, as we mentioned earlier, we want to be able to test (including screenshot tests) the application not only in positive, but also in negative scenarios (internet lost, server returned an error, record not found in the database, subscription or license expired). Because Kaspresso is a native framework, we can emulate any state inside the tests. Besides, having implemented Screen classes in regular E2e tests, you can reuse them and write screenshot tests very quickly.

Secondly, Kaspresso allows you not only to set all the states by emulating various scenarios, but to do it directly without even having to launch the application! You can simply take the required Activity or Fragment and put it into the special holder.

private lateinit var scenario: 
FragmentScenario<ScreenshotSampleFragment>

@ScreenShooterTest
@Test
fun test() = before {
  scenario = launchFragmentInContainer()
}.after {
}.run {
  scenario.onFragment {
    val view = getUiSafeProxy(it as ScreenshotSampleView)

    view.setNoInternetState()
    captureScreenshot("No Internet state")

    view.setContentLoadingState()
    captureScreenshot("Content is loading state")

    view.setServerErrorState()
    captureScreenshot("Server error state")
  }
}

For more details, have a look at this sample.

Thirdly, Kaspresso allows you to change languages both at the system level and at the application level (by changing string resources in the application). This gives you more options and allows you to run tests faster

Finally, Kaspresso provides the captureFullWindowScreenshot method for screenshotting long screens without having to scroll, as well as the DocLocScreenshotTestCase base class, which you can extend for writing screenshots tests.

Detailed and understandable logs and reports

Thanks to internal interceptors, Kaspresso wraps the test code, adding detailed logs indicating each step with screenshots. After running the tests, various artifacts are available (logs, screenshots, hierarchy of graphic elements, on-screen string identifiers, and much more, which can be downloaded automatically at the end of the run). For visual clarity, here is an example log:

Try the open source framework Kaspresso now with our step-by-step tutorial in codelab format

We recently published 14 lessons about Kaspresso to help you understand better and get real practice. All the lessons are based on the real and ready application, which you can run on your device or emulator (you need to download the source code from our GitHub and run the Tutorial project). If you experience problems during any lesson, you can switch to the TECH-tutorial-results branch and see the final implementation of all tutorial tests.

Tutorial structure

The Tutorial is divided into steps (lessons). Each lesson begins with a brief overview and ends with a summary and conclusions.

How to study this tutorial?

We strive to make the lessons independent from each other, but this is not always possible. For a better understanding of Kaspresso, we recommend starting with the first lesson and moving sequentially to the next. With the codelab format you will combine theory and practice, repeating the instructions from the lessons step by step. In the Kaspresso project, in the tutorial folder, there is an example of the application code for which tests will be written. The first lesson tells you how to download it.

What do you need to know to complete the Tutorial?

We are not trying to teach you autotests from scratch. At the same time, we set no restrictions on knowledge and experience for passing the tutorial, and keep the narrative understandable to autotest and Android beginners. It is almost impossible to talk about Kaspresso without terms from the Java and Kotlin programming languages, the Espresso, Kakao, UiAutomator and other frameworks, the Android operating system and testing itself as an IT area. Nevertheless, the main focus is on explaining Kaspresso itself, and wherever terms are mentioned, we share links to official sources for detailed information and better understanding.

Feedback

If you find any typos, errors or inaccuracies in the materials, want to suggest an improvement or add new lessons to the Tutorial, you can create an Issue in the Kaspresso project or open a Pull request (materials from the Tutorial are in the public domain in the docs folder).

If the Tutorial did not answer your question, you can search the Wiki section or Kaspresso in articles and Kaspresso in video.

You can also join our Discord channel and ask questions there.

I’m not saying goodbye

As you can see, Kaspresso has a lot of very useful functionality that will help simplify your test-writing process. The structured step-by-step tutorial in codelab format lets you not only study this framework, but learn how to write autotests in general.

We will publish a series of articles here, so follow our blog.

Detailed info on how to try out Kaspresso is available on our GitHub page and website. Give stars to our project on GitHub and join our Discord community.