This text is to tell one reason that, if you have access to the source code of your mobile app, you should stop using Appium and use the native testing framework instead.
Context
Most of the mobile projects I worked on did not have a complete mobile CI pipeline due to the costs involved in building/renting a device farm. Only in 2 projects the mobile tests ran on CI: one the CI already existed and the other we built during the project, both running on a Mac Mini, and not in an automatic way.
What usually happened is that I coded my tests, using Appium with Ruby, and ran them on my own machine. The problem is that putting an Android emulator and an iOS simulator running at the same time practically means you can't use your computer for anything else.
In this situation, the tests only ran if I ran the tests.
In the project that gave the idea to write this text, the Devs did not do unit tests and the “it's ready, we just need to test it” was often said. So all responsibility for testing relied on me and the other QA.
As running the tests on the emulator and simulator at the same time “locks” the machine, it ended up that there was only automation code, but the tests were done manually.
The idea
One day, while writing testing code, I wasn't able to make Appium click or validate a field on iOS at all. Then I thought: “enough! This responsibility cannot be only mine!” That happened during the morning.
In the afternoon I began to look for how to automate UI with Swift and found this course from Test Automation University (TAU), from Aplitools. I spent this afternoon learning XCUITest. The course has 1h30min. I must have taken about 3 hours to finish.
The next day I automated a feature as an example and called the dev to talk. Since everything was already done in Ruby, it was just a matter of “translating” it into Swift.
What would be the idea: the idea is, since the tests will be at the same project, in the same repository, when the developer finishes the code, play the tests before push the code. If a test broke, we could look together at what was happening and fix the problem.
Even though I was still responsible for coding the tests, at least the tests would run, even with manual triggering. And I wouldn't be the only one responsible for that. I would no longer need to worry about automated scenarios.
After a while I was already being asked when it took me a while to merge a test PR! We achieved 73% coverage on iOS practically overnight!
Of course, I also looked for Espresso courses for Android. And I also found it at TAU.
Appium X Native Automation
Advantages of Appium
- Several languages (Ruby, Java, Python…)
- 1 code, 2 platforms (or 3, if we consider web)
- Test on the apk/app, which is what the customer will receive
- Automation of “the entire device”, for example camera, gallery…
- Easier to get material about E2E testing
I was reading that if you put a security tool, which I don't remember the name of, in the application, it is mandatory to test it on the apk/app, because directly in the code it is not reliable.
Disadvantages of Appium
- “Segregated” QA
- One responsible person
- Additional integration required, as the code is in another repository
- Slower (up to 8x)
Disadvantages in my context
- It is necessary to build the apk/app on the QA machine
- Run automation manually and locally, since we didn't have a CI
- Free CIs do not include the diversity needed for mobile testing
- “Impossible” to use the computer while running mobile tests
Remembering that these lists are based on my experience. For you it may be different.
Native automation advantages
- “Dev” language
- You “guarantee” that the tests will be run (we have to trust our team mates)
- Split testing responsibility
- More interaction and exchange between Devs and QAs
- “Easier” to run locally
Disadvantages native automation
- Learning and dealing with two (or three) different languages at the same time
- Maintenance in two (or three) codes
- Automation only in the context of the application. Camera and other interactions need to use UIAutomator, in case of Android.
Talking to other QAs who use this approach and reading some articles on the internet, the advantages outweigh the disadvantages. And I'm seeing it.
Getting started with native automation is just the beginning. In the near future I want to go deeper in the test pyramid and help with instrumented and unit testing.
And you, what do you think of the idea of migrating your tests to the development language? Comment here!
And share this post so more QAs see that it's cool to be closer to development.
And keep Testing Softly!
Top comments (0)