In this post, I’ll go over the different factors and scenarios that impact a mobile app’s performance. I’ll also explain how to test mobile app performance in the different ways that matter including which tools and metrics should be involved.
You’ve probably interacted with an app on your phone or tablet that’s slow, takes a long time to load, freezes or even crashes on you altogether.
Frustrating, right?
On the flip side, we’ve all experienced apps that, from day one, have never given us any trouble and we love them for working so fast and effectively.
Or, we haven’t even thought about it all because isn’t that how they should be?
What causes one app to be crash-prone and another, fast and reliable?
Whether an app has good or bad performance depends on three factors combined: the backend, the network, and the app itself running on the device.
A developer or mobile tester can measure the performance of an application in different scenarios.
For example, they can test for when there’s a concurrency of users on the app at the same time, on different devices (which vary in hardware resources and screen sizes), and multiple networks such as 3G, 4G, Wifi, and more.
The reality is that there are many variables that affect the performance of a mobile application. Moreover, a user may have a very bad experience with your app and the cause might not even have anything to do with the code or its implementation.
But, by running performance tests for each of these three factors, you’ll be able to identify problems and optimize your app for the best user experience possible.
Keep reading as we’ll cover the different types of tests for each factor, what to measure, and what tools are available to help you along the way.
1st mobile performance factor: the backend
A mobile app’s backend architecture is generally based on an application server, a web server and a database.
When it comes to the backend, the things related to performance that are important to know when an app is under load are the server’s response times, database queries times, and the server’s resource usage.
Using this information, it’s easier to detect issues such as:
- High server response times
- Bottlenecks or breakpoints in the database and application server resources
- Poor implementation of escalation policies
So what kind of tests are normally run to check the app’s backend performance? Load tests.
This is when you simulate load on the backend in different ways, whether it be through stress testing, peak testing, endurance testing, load testing, etc.
In general, the objective of these tests is to understand how the backend systems of an app behave and handle a certain volume of concurrent users.
There are several tools that allow you to load test your mobile app. The most commonly used ones include:
Apache JMeter – the number one open source load testing tool
Gatling – a developer-friendly, open source load testing tool with scripts written in Scala
BlazeMeter – a cloud performance testing platform that scales your JMeter or Gatling tests for a greater amount of concurrent users
2nd mobile performance factor: the network
With regards to the network that the device is connected to, there are two key things to measure: latency and bandwidth.
Latency is the time that elapses when information is sent on the network (measured in milliseconds).
Bandwidth is the maximum capacity (the amount of data) that can be transmitted through the network (measured in bits per second).
For mobile performance, the lower the latency and the higher the bandwidth, the better.
An app’s performance can vary depending on, for example, whether it’s connected to a 3G network or a 4G network, and unfortunately, this is beyond an app developer or tester’s control.
But, it is possible to incorporate the network during the mobile app performance testing process, simulating the different types of networks and measuring their impact on the response times, both on the server side and the client side.
Luckily, the third factor is something that’s been overlooked for years and we now have ways to incorporate it into our test cycles and gain greater control over it.
3rd mobile performance factor: the client
For a mobile app, the client is the software that runs on the device, the app itself.
With client-side testing, the variety of devices, operating systems and screen sizes of a device that an app is running on comes into play.
Today there are thousands of mobile devices on the market which vary by brand, hardware and operating system.
And each year, it becomes more and more difficult to ensure that an app works well on most of the devices on the market because the amount only continues to grow.
This image below shows the different Android versions in use over the past seven years:
And here are the different versions that were used in just 2019:
In 2019, you needed to test on at least 6 different devices with OS’s going from Lollipop 5.0 to Pie 9.
Fortunately, unlike Android developers, iOS developers’ lives are a bit easier as the iOS community consists of fewer devices and operating systems.
Testing client-side performance allows you to understand how the app behaves on a device and how it uses the resources that it shares with other apps.
Running tests for client side performance, you’ll also be able to find areas to improve that you do have control over (unlike the network), resulting in a better user experience.
It's critical that any mobile app development team integrate client-side performance testing into its test cycle.
There are several areas with different metrics you can gather to understand client side mobile app performance:
Device resource usage:
- % CPU
- Memory (Mb)
- Data sent and received by the application (Kb)
- % Battery usage
Rendering:
- Render time (ms)
- Frames per second (fps)
- Render lag (janks)
Errors:
- Exceptions
- ANR (Application Not Responding) in Android
Transaction response time, which can also be measured from the user perspective but involves:
- Server response times
- Database response time
- Network latency
- Screen drawing time
Once you’ve got all of this information, you can uncover critical problems such as:
Memory leaks: when an app doesn’t free up memory well, it increases until it runs out of available memory and then crashes
Screen freezes: due to very high rendering times
Crashes: exceptions that jump in the app and cause it to suddenly close… bummer!
For native apps, there are profiling tools such as Android Studio and Instruments (for Android and iOS respectively).
These tools are of great help when debugging an app and finding the root of certain kinds of performance problems.
In addition to Android Studio and Instruments, you can also use Apptim, the tool we've launched at my startup to give entire development teams access to mobile app performance information and KPIs, where anyone can quickly and easily run tests.
With Apptim, you can test all of your native apps’ performance with just one tool, instead of one for Android and one for iOS and also compare performance over time.
Test mobile app performance as early as possible
Now, you may be wondering, “At what point in the development cycle should I start running client-side performance testing?”
If you ask me… start as soon as possible.
If you start testing early on, from the first version of your app, you’ll give yourself more time to understand how it behaves, get familiarized with client side metrics, and run these tests alongside functional ones in each sprint.
Another advantage of testing early is you can test on different devices at your disposal as well as understand how these metrics evolve over time.
It’s important to understand how the metrics change over time so you can identify what changes to the app alter its performance (find performance regressions) and have more time to remedy degradations.
This helps a developer understand how the decisions they make affect performance and ultimately, avoid disappointing users when they find that the app is running slower than usual.
I hope this post provided you an overview on how to test mobile app performance!
Have any questions? Leave a comment!
Top comments (0)