DEV Community

Cover image for Making Android UI testing enjoyable
Rafa Vázquez for Adevinta Spain

Posted on

Making Android UI testing enjoyable

UI testing in Android has always been controversial for many reasons. Tests are slow because they must run in emulators or real devices. They can be flaky for many different reasons. Test code is usually hard to read and maintain because of Espresso and Android APIs. UI testing is a pain in the ass, and it will continue to be, I’m sorry.

But still, in my experience, these kinds of tests are the most useful ones for Android development. While unit tests help us during development, UI tests give us peace of mind during releases, knowing that the most important features of the app won't just stop working for everybody.

In this post, I want to share some of the work we’ve been doing at InfoJobs the past few years to make UI testing less painful.

Our main efforts went to

  • Reduce flakiness
  • Make tests more semantic
  • Write simpler and maintainable tests

Mandatory disclaimer: There are many approaches you can take to UI testing, and most of them are right. I’m not going to tell you how you should do it. I’m going to show what has worked for us and how we do it, so you can hopefully take something from it.

Beginning with UI tests and Espresso

Espresso is Google's official UI testing framework. It can perform actions and check conditions on views. So a really simple UI test could look like this:

@Test
fun mySimpleTest() {
  onView(withId(R.id.greet_button)).perform(click())
  onView(withText("Hello Steve!")).check(matches(isDisplayed()))
}
Enter fullscreen mode Exit fullscreen mode

But in our day-to-day, we want to cover more complex features and flows involving multiple screens. So a real test could become something like:

@Test
fun shouldShowRecommendedOffers_whenComingToHomeBackAfterLogin() {
  // Perform search
  onView(withId(R.id.toolbar)).perform(click())
  onView(withId(R.id.keyword_field)).perform(replaceText("Java"))
  onView(withId(R.id.search_button)).perform(click())
  // Log in
  onView(withId(R.id.searchResultLoginButton)).perform(click())
  onView(withId(R.id.loginEmailField)).perform(replaceText("email@example.com"))
  onView(withId(R.id.loginPasswordField)).perform(replaceText("password"))
  onView(withId(R.id.loginButton)).perform(click())

  // Go back
  onView(isRoot()).perform(pressBack());

  // Assert header text
  onView(withText("10 ofertas recomendadas")).check(matches(isDisplayed()))
}
Enter fullscreen mode Exit fullscreen mode

It’s not the prettiest code, but it works. This is our starting point. Let me tell you what we did to improve it from here.

Barista

Espresso is ugly

A bit of history. When we started using Espresso to write our UI tests years ago we had to learn its API. And it wasn’t pretty. Don’t get me wrong, it’s very powerful and flexible. We can do almost anything to views. But it’s not pretty. The code above is barely readable.

Tip: Check out this awesome cheat sheet in the official docs to understand how the Espresso API works.

At the time, a couple of backend engineers were learning Android development and working on some features for the app. They wanted to cover the features with UI tests, but having to learn and remember the syntax was a problem. So they suggested adding static methods for common actions that they used over and over again. Like clickOn(R.id.button) and assertDisplayed(“Some text”).

I must confess that I didn’t agree with this idea at the time. I believed that we should get used to the Espresso API so we could understand how it works and do more complex things when needed. Luckily my friend Roc was also on the Android Team and convinced me to play along. So we tried it for a while. And oh boy, was I wrong. It was a great decision, it improved a lot the readability of our tests.

Espresso is dumb

Ok, maybe “dumb” is unfair. Let’s say it’s “strict”. For example, we would often need to interact with elements inside scrolls. Espresso fails if you perform an action on a View below a scroll. So we need to tell it to scroll first.

Scroll click

fun click(@IdRes id: Int){
  onView(withId(id)).perform(scrollTo(), click())
}
Enter fullscreen mode Exit fullscreen mode

But there is a catch! Espresso will fail if the view is not inside a ScrollView. Bummer!

From the point of view of the test, I don’t care if the view is inside a scroll or not, I just want the test to click it as if it was an user using the app. So we came up with a trick:

fun click(@IdRes id: Int){
  try {
    onView(withId(id)).perform(click())
  } catch (e: PerformException) {
    onView(withId(id)).perform(scrollTo(), click())
  }
}
Enter fullscreen mode Exit fullscreen mode

Disclaimer This is a naïve approach, but it's a quick-win. It can be improved in many ways, but it was our first iteration and it worked for us.

Another common example is having a ViewPager with the same layout on multiple pages.
ViewPager

A simple click() action will fail with

androidx.test.espresso.AmbiguousViewMatcherException: 'with id: id/button' matches multiple views in the hierarchy.
Enter fullscreen mode Exit fullscreen mode

As you see, Espresso is strict. If it gets confused and doesn't know what to do, it will fail. It forces you to be more explicit about your actions. There's nothing wrong with that, it's a valid approach. But we were comfortable assuming that, if there are multiple matching views, we only want to click the one that is displayed:

fun click(@IdRes id: Int){
  // I omitted the try/catch from the previous example for brevity
    onView(allOf(isDisplayed(), withId(id))).perform(click())
}
Enter fullscreen mode Exit fullscreen mode

This kind of code could be considered a bad practice in production code. But in testing code we found it makes our tests simpler and more robust since they would not depend on variable things like the screen size or future layout changes. It was totally worth it!

With this same philosophy we created a class named EspressoUtils, with many more similar static methods. It kept growing as we added new utilities for new scenarios we wanted to test. Over time we noticed that even long tests were much more readable using our custom methods than the Espresso API, but more importantly, easier and faster to write.

We added click(id), click(stringRes), openDrawer(), acceptDialog(), assertDisplayed("String"), and many more. Writing new UI tests with EspressoUtils was a delight. We would copy and paste this file internally from project to project until we realized we should share it with everyone as a library. So we moved it to an independent project, renamed it to Barista (great name Sergi!), designed a beautiful logo (thanks Diego!), added tests for all the interactions and assertions (thanks Roc for so many hours!) and published it. Barista was born!

Introducing Barista

Using Barista we removed a lot of the boilerplate from Espresso, making the code easier to read and write. Time has passed and Barista has grown a lot. Today it contains many more interactions and assertions. By making acceptable assumptions it is smarter, it can decide to scroll the screen when needed, it can deal with ViewPagers that contain similar layouts, it interacts with ListView and RecyclerView indistinctively, it allows making custom assertions with a couple of lines thanks to assertAny, it contains some useful Test Rules to fight flaky tests like cleaning Shared Preferences and Databases between tests, and more. Take a look at the Readme, you will be surprised by how much stuff it can do with little code.

GitHub logo AdevintaSpain / Barista

☕ The one who serves a great Espresso

Barista

The one who serves a great Espresso

Travis Download Hex.pm

Barista makes developing UI test faster, easier and more predictable. Built on top of Espresso, it provides a simple and discoverable API, removing most of the boilerplate and verbosity of common Espresso tasks. You and your Android team will write tests with no effort.

Download

Import Barista as a testing dependency:

androidTestImplementation('com.schibsted.spain:barista:3.7.0') {
  exclude group: 'org.jetbrains.kotlin' // Only if you already use Kotlin in your project
}
Enter fullscreen mode Exit fullscreen mode

You might need to include the Google Maven repository, required by Espresso 3:

repositories {
    google()
}
Enter fullscreen mode Exit fullscreen mode

Barista already includes espresso-core and espresso-contrib. If you need any other Espresso package you can add them yourself.

We are super proud to have reached 1.2K stars in GitHub and 1 million downloads in Bintray. And it makes us especially happy seeing a lot of people in the community using, contributing and enjoying it.

But the story doesn't end here. Keep reading to see what else we did to improve our tests.

Page Objects

Barista was just the tip of the iceberg. It’s not just a useful library. It changed our approach to UI testing, it gave us the mindset of writing simpler and easier to read tests. Page Objects are a step in a similar direction, they make test code more semantic and easier to write.

The idea of using Page Objects started many years ago. QA engineers and web developers were used to a concept named “Fragment”. Basically, they were Java abstractions over the actions and assertions that could be done on each page from the perspective of the user (yea, not those fragments). When some backend engineers started working on the Android app they imported those ideas to our shiny Espresso tests. We later learned (thanks to Pedro) that this pattern was already a thing called Page Object. Great! It wasn’t a bad idea after all! 😃

pageObject
(image from https://martinfowler.com/bliki/PageObject.html)

Our implementation of these Page Objects has evolved over time. We now call them Screen Objects, but that's not relevant. After many iterations and agreements among our Android Team, the result looks like this:

@Test
fun shouldShowRecommendedOffers_whenComingBackAfterLogin() {
  HomeListScreenObject()
    .clickSearchToolbar()
    .search("Java")
    .clickFloatingLoginButton()
    .fillCredentialsAndLogIn()
    .assertThat { headerContains("10 ofertas recomendadas") }
}
Enter fullscreen mode Exit fullscreen mode

Example implementation (simplified)
class HomeListScreenObject : NavigationScreenObject() {

  fun clickSearchToolbar(): SearchScreenObject {
    clickOn(R.id.toolbar)
    return SearchScreenObject()
  }

  fun assertThat(assertionBlock: HomeListScreenAssertions.() -> Unit): HomeListScreenObject {
    HomeListScreenAssertions().assertionBlock()
    return this
  }
}

class HomeListScreenAssertions : BaseScreenAssertions(R.id.homeListRoot) {
  fun headerContains(text: String) {
    assertDisplayed(text)
  }
}

class SearchResultScreenObject : BaseScreenObject {

  fun clickFloatingLoginButton(): LoginScreenObject {
    clickOn(R.id.searchResultLoginButton)
    return LoginScreenObject()
  }

  fun clickFilters(): SearchFiltersScreenObject {
    clickOn(R.id.searchResultFilterButton)
    return SearchFiltersScreenObject()
  }

  fun assertThat(assertionBlock: SearchResultScreenAssertions.() -> Unit): SearchResultScreenObject {
    SearchResultScreenAssertions().assertionBlock()
    return this
  }
}

class SearchResultScreenAssertions : BaseScreenAssertions(R.id.searchResultRoot) {

  fun createAlertButtonIsDisplayed() {
    assertDisplayed(R.id.searchResultCreateAlertButton)
  }

  fun searchBarContains(text: String) {
    assertAnyView(
      viewMatcher = withParent(withId(R.id.toolbar)),
      condition = withText(text)
    )
  }
}

Enter fullscreen mode Exit fullscreen mode

The test code can be read out loud to any person following those steps in front of the app and they would understand what to do. You don’t see any Espresso code or Android resources ID. You only see concepts from the user perspective.

Here are some general rules that we follow:

  • One PageObject per screen: There is a Page Object class for each screen in our app. The concept of Screen is subjective, it can be an Activity, a Fragment inside the BottomBar, a complex dialog, etc. When in doubt we ask ourselves: “what does the user perceive as a screen in this case?”.
  • Actions: A PageObject contains functions for the actions that the user can perform on the current screen. They can be very concrete like clickSaveButton(), or wider like fillFormFields(). They can also contain parameters, like search("Java").
  • Navigation: Each function returns the PageObject of the next screen after the action is performed. If the user stays on the same page, it returns itself. If the user navigates to a different screen, it returns the PageObject of that screen. This way we can fluently write actions across multiple screens, with the help of the IDE’s autocomplete.
  • Assertions: We have an assertThat builder function in every PageObject that returns the Assertions of this PageObject. This Assertions object contains functions to assert different things on the screen.
  • Be pragmatic: We don’t write every possible action or assertion for every screen. We add them as we need them. Remember that this is still code that must be maintained.

No fancy framework here, we try to keep it as simple as we can. These abstractions let us write UI tests really fast and keep them very semantic and readable. Modifying a feature or testing a new one is a breeze.

Barista is hidden in the implementation of the PageObject functions, so we still benefit from its magic. Sometimes we'll implement more complex actions or assertions that cannot be written with Barista and we'll write some custom Espresso code. Barista and Espresso are not exclusive!

Utilities

The last piece of our UI testing story is a series of smaller utilities that we've been adding over time. We grouped them in a InstrumentationTest interface, and I'm going to show some of them because I consider they help a lot with test readability.

Launch

This function takes an Intent and a lambda block, launches the Activity with the Intent, runs the code from the lambda, and then closes the Activity.

If it sounds awfully familiar to an ActivityScenario, it is because our launch function is just a wrapper around ActivityScenario to make the code a little bit easier to read and write.

@Test
fun openHome() {
  givenCountryIsSelected()
  givenUserIsNotLogged()

  launch(intentFactory.home.create(appContext)) {
    assertDisplayed(R.id.homeOffersList)
  }
}
Enter fullscreen mode Exit fullscreen mode

Same code without the alias
@Test
fun openHome() {
  givenCountryIsSelected()
  givenUserIsNotLogged()

  ActivityScenario
    .launch<Activity>(intentFactory.home.create(appContext))
    .use { scenario: ActivityScenario<Activity> ->
      assertDisplayed(R.id.homeOffersList)
    }
}
Enter fullscreen mode Exit fullscreen mode

fun launch(intent: Intent, testBlock: ActivityScenario<Activity>.() -> Unit) {
  ActivityScenario.launch<Activity>(intent).use { scenario ->
    scenario.testBlock()
  }
}
Enter fullscreen mode Exit fullscreen mode

Before this, we were using the old ActivityTestRule (now deprecated). The test code didn’t look too bad, but it had some design issues that have been solved thanks to ActivityScenario. To mention some of its advantages, it provides fine-grained control of when the Activity gets launched and destroyed, we can easily launch different Activities on the same test, it lets you interact with the Lifecycle and even run code on the Activity’s Thread without sketchy tricks. There is an ActivityScenarioRule, but I would personally recommend avoiding it in favor of ActivityScenario, in order to have better control over the Activity or Activities in your test.

DeepLinks

Small and silly trick, but we added a launchDeepLink function to simplify testing deeplinks, hiding boilerplate code and workarounds.

@Test
fun offerSearchOpensWithDeepLink() {
  launchDeepLink("https://www.infojobs.net/ofertas-trabajo/barcelona") {
    SearchResultScreenObject()
      .assertThat {
        screenIsDisplayed()
        searchBarContains("Barcelona")
      }
  }
}
Enter fullscreen mode Exit fullscreen mode
fun launchDeepLink(uri: String, testBlock: ActivityScenario<Activity>.() -> Unit) {
  val deepLinkIntent = Intent(Intent.ACTION_VIEW, Uri.parse(uri))
  // Component required because of a bug in ActivityScenario: https://github.com/android/android-test/issues/496
  deepLinkIntent.component = ComponentName(appContext.packageName, DeepLinkActivity::class.java.name)
  return launch(deepLinkIntent, testBlock)
}
Enter fullscreen mode Exit fullscreen mode

IntentFactory

Another common task when dealing with Activities is creating Intents. In our app, we have an IntentFactory class what we use to create Intents to any screen. This class is injected by Koin, because it might depend on other dependencies like Feature Flags or A/B tests.

By having a commong InstrumentationTest interface we provide an easy shortcut instead of manually injecting it on every test eveery time. Again, a very small improvement that adds up!

interface InstrumentationTest {
  val intentFactory
    get() = KoinContextHandler.get().get<IntentFactory>()
  // ...
}
Enter fullscreen mode Exit fullscreen mode

Rules

I'm sure you usually apply some JUnit rules to your instrumentation tests, don't you? We can apply a bunch of common test rules to all our tests. And we have many of them!

@get:Rule
val instrumentationRules: TestRule
  get() = RuleChain.outerRule(
    FlakyTestRule().allowFlakyAttemptsByDefault(FLAKY_ATTEMPTS))
    // ↓ All rules below flakyTestRule will be repeated if the test fails
    .around(FinishAllActivitiesRule())
    .around(DisableAnimationsRule())
    .around(IdlingUseCaseRule())
    .around(ReloadKoinRule())
    .around(ClearFilesRule().includeFilesMatching("/(.*).json"))
    .around(ClearCacheRule())
    .around(ClearPreferencesRule())
    .around(ClearDatabaseRule())
    .around(ResetLocaleRule())
    .around(RESTMockRule())
    .around(ScreenshotOnFailureRule())

private val FLAKY_ATTEMPTS: Int
  get() = if (BuildConfig.IS_CI) 7 else 1
Enter fullscreen mode Exit fullscreen mode

And more!

I can't list every single little helpful feature we have for our tests, but I'd say these are the most impactful ones. Mocking the API, testing custom views, testing event tagging or screenshot testing to name a few. Maybe we'll share some more of these in the future!

The takeaway from this last part is that there is a lot of verbosity associated with UI tests that can be hidden away to make them easier to read, write and modify. These small things only save a few lines of code in each test. But they work together to make you forget about the testing framework, focus on what you really care about in the test, and let the existing infrastructure handle the rest.

Conclusion

UI tests can be a pain sometimes. But investing some time into them can make your life much easier, you'll soon find yourself writing new tests without effort.

I showed many different things here. You might like some, you might dislike others. I told our story because context is important. Find what works best for you and your team, and don't be afraid to iterate your solutions.

Writing a test for a button: 4 minutes.
Finding why the button hasn't worked in production for a week because you didn't write that test: 4 hours.
Write more UI tests! Don't trust Android!


Last but not least, I wrote this post but the contents and learnings are a product of many people that have worked on our Android Team. Thank you Roc, Jose, Rubén, Sergi, Bernat and many more contributors!

If you have any questions about the post or want me to expand on some part, please feel free to leave a comment below or ping me on Twitter @sloydev.

Thanks for reading!

Latest comments (11)

Collapse
 
matzuk profile image
Eugene Matsyuk

Hi guys!
Barista is an interesting solution to make Espresso tests easier. But, there is a more modern and perfect solution - Kaspresso. Check it =)

Collapse
 
liweijian profile image
liweijian

Hi, I was wondering could you elaborate on why you think Kaspresso is more modern and perfect?

Collapse
 
anticafe profile image
anticafe

I also see Kakao and Kaspresso as a good library to support write UI test. How does it compare to Barista?

Collapse
 
sloydev profile image
Rafa Vázquez

I haven't personally used Kakao or Kaspresso, so take my response with a grain of salt. But from what I've seen about them, they seem more opinionated than Barista on how to write your tests. They are very linked to the concept of page object with a custom DSL, which can be helpful for some projects for sure.

Barista is more "low level", closer to Espresso but with a simpler API and some hidden "magic" as described above. You decide how to structure your own tests. In this post, I showed how we do our PageObjects, but you're free to use any other approach.

In my personal and debatable opinion, our PageObjects are more semantic, because we write them from the perspective of what the user would be doing. However, the DSL from Kakao exposes some information about the structure of the screen (button this, recyclerview that). I'm not saying that's wrong, it's just different.

I hope I could shed some light on your question :)

Collapse
 
garispe profile image
guillermo.arispe

Great job!

I want use Barista, but I have an issue:

Library Version:

3.7.0

Describe the Bug:

I followed the instructions to integrate Barista. I put this lines in my build.gradle

androidTestImplementation('com.schibsted.spain:barista:3.7.0') {
exclude group: 'org.jetbrains.kotlin'
}

But when I try to use it, Android Studio don't recognize the library.

What is my error?

I'm using Android Studio 4.1 and com.android.tools.build:gradle 4.0.1

Collapse
 
sloydev profile image
Rafa Vázquez

I see in Github you already solved it :)

Collapse
 
liviufromendtest profile image
Liviu Lupei

Nice article.

You should also try Endtest for building and executing automated tests for Mobile Apps.

It uses an Appium engine and it even has a test recorder.
These chapters are a good starting point:
How to create Mobile Tests
Finding elements in Mobile Applications
How to execute Mobile Tests

I actually work there, so feel free to ask me anything about it.

Collapse
 
liweijian profile image
liweijian

If Endtest is using Appium, I believe that's another topic. According to the official doc

Appium's flagship support for automating Android apps is via the UiAutomator2 driver

Espresso is much more fast, and have lots of other advantages like very good at test RecyclerView.

Collapse
 
liviufromendtest profile image
Liviu Lupei

Do you have any studies or data to support your claim that Espresso is faster than UiAutomator2?
I'm curious to learn more about that.

Thread Thread
 
liweijian profile image
liweijian • Edited

From google official training course

Espresso tests run optimally fast! It lets you leave your waits, syncs, sleeps, and polls behind while it manipulates and asserts on the application UI when it is at rest.

From 3rd introduction

In my opinion, the first trait is Espresso faster than Ui Automator.

Thread Thread
 
liviufromendtest profile image
Liviu Lupei

Espresso was built by Google.
Your first source is literally from Google.
And your second source is an article published on Medium by someone.

None of those articles contain any comparison data.

I was hoping for something more reliable, that would include data, measurements, comparisons, etc.