What is End to End Testing?
End-to-end (E2E) testing is a technique that tests the entire software product, as well as all of its systems
(Katalon, 2019).End to end testing is vital to mobile app and software development, as it allows the software to be tested in a simulated, real life scenario (Software Testing Help, 2022).
The main purpose of E2E testing to test the application from the outside, just like a user would, without knowing anything about the internal functions of the software. This makes E2E tests a form of black box testing.
Another feature of E2E testing is that it tests lots of different parts of the system to make sure that they are all working together once connected, this is known as integration testing.
Why E2E Testing?
E2E tests assess the whole system and the software in its entirety, examining multiple subsystems together, all with one single test. For example, an E2E test would examine the user interface, interaction, navigation and API (unless mocked) and it will act as an integration test.
As well as offering an examination of the full system, E2E tests are the most effective way of testing how a user will use an application, without requiring a real life user. High-priority user flows, such as signing up or creating a booking, have to be repeatedly scrutinised, in order for the application to work correctly. This can be time consuming and repetitive. However, with the use of an E2E test, the flow can be simulated and interacted with in the same way that a user would, whilst verifying that all parts and subsystems within that flow are working as they should.
E2E testing is a great way to establish that your application is working, without having to put in the time to manually test.
E2E Testing Setbacks
Although E2E tests will test many subsystems in one test, it will not fully test these subsystems, only the parts used in the test. It’s also possible for the same test or user flow to fail for a scenario.
It can be very hard to tell what the problem is when a test fails, as the error could be in any part of the whole system being tested.
It’s hard to cover edge cases in E2E tests as this will take a lot of time to run the test and possibly restarting or reinstalling the app. E2E tests can also be unreliable, with the setup often being difficult and time-consuming
These tests are bound by the app limitations, the same as a user. This includes waiting on the loading screens, having to login before testing, using the API and only seeing onboarding screens once. Whilst there are a number of setbacks with E2E testing, the benefits do outweigh the setbacks, and the opportunities for E2E tests are impressive.
What can we do with E2E tests?
Although many E2E tests will consist of simple instructions, there are many interactions with the app and device that can be simulated through E2E tests.
A simple test will usually consist of:
- Pressing a button
- Typing text into a text box
- Waiting for a new element or screen to be shown
However, with the use of other tools, more specific and more complex tests can be run. For inspecting elements, you can read text, element size, css style and tell if an element is visible on the device's screen.
For touch interactions, you can tap, double tap, type, pinch, scroll and flick.
For finding elements, you can look for a group or list of elements, this means you can iterate through every element in a list and set the test to interact with all of them.
Using Appium lets you write these tests in your chosen programming language. This also lets you combine these tests with if statements and loops, making it possible to have more complex tests that can handle different situations.
Device interactions are also possible. It’s possible to set the internet speed to test specific flows and features with a slow internet speed. You can also toggle airplane mode to see how your app performs when offline.
Other device interaction you can use include simulating shaking the device, locking / unlocking the device and accessing clipboard. You can also add and delete files.
The scope of the E2E test is almost without restriction, as long as you know how to code it!
What is Appium?
Appium's philosophy documentation summarises what Appium is pretty well:
Appium is at its heart a web server that exposes a REST API.
It receives connections from a client, listens for commands, executes those commands on a mobile device, and responds with an HTTP response representing the result of the command execution.
The fact that we have a client/server architecture opens up a lot of possibilities: we can write our test code in any language that has a http client API.
Limitations of E2E testing
One limitation is that to easily interact with any UI element in your app, a prop must be given to the view, most used is the "accessibilityLabel" prop, in which every element should have a unique ID.
Your tests are bound by the same rules as a user is. This includes having to log into the app before being able to access other screens, some screens not being viewable for certain types of users and only being able to view onboarding screens once. There are ways to get around this, by re-installing the app or by logging in with a different account.
E2E tests take a long time to execute, compared with other types of tests and they are hard to setup and can be very flaky.
It's hard to find what causes an error when a test fails. And lastly, time is needed for the maintenance of these tests, as the app changes. This requires a developer who is both familiar with the app and the E2E tests.
Future Work
One of the best features of E2E tests is that they can be run as part of your CD/CI pipeline, in the cloud, using a cloud device service, of which there are many options.
This lets you have these E2E tests executed every time you want to merge a branch or after running a build to give you much higher confidence when merging or releasing a build.
Whilst E2e testing has many benefits, the ideal testing solution includes a full test suite of unit, integration and regression tests combined with dependency injection. This is the only way that you can have full confidence in your tests.
Top comments (0)