Forem

Cover image for Discovering UI Performance testing with XCTest. Navigation performance.
Dmitrii Morozov
Dmitrii Morozov

Posted on

3 1 1

Discovering UI Performance testing with XCTest. Navigation performance.

One of the most important components of app performance is the performance of navigation transitions between different screens. Applying new changes to a screen without proper testing can significantly degrade this aspect of an app’s performance. In this article, I will show how to build performance tests to address this problem. This article has some references to my previous article about scrolling performance and I recommend to read it first.

Setup

I prepared the demo project. This is a simple app that shows you a list of plants, you can select a plant and open the details screen. For all measurements I used MacBook M2 Pro with Ventura 13.0 and iPhone 13 Pro with iOS 16.0, XCode version is 14.2.

Here is the test I used for testing. It tests navigation performance when opening the details screen from the main screen:

import XCTest

final class NavigationPerformanceTests: XCTestCase {
    let app = XCUIApplication()

    func testNavigationTransition() {
        app.launch()
        let measureOptions = XCTMeasureOptions()
        measureOptions.invocationOptions = [.manuallyStop]

        measure(metrics: [XCTOSSignpostMetric.navigationTransitionMetric], options: measureOptions) {
            app.staticTexts["Monstera"].tap()
            app.navigationBars.buttons.element(boundBy: 0).tap()
            stopMeasuring()
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

As I found out in the previous article there are limited metrics available on a simulator compared with a real device. To understand if this method is applicable I will test it both on a simulator and a real device.

Testing

Firstly we need to set up baselines for our test. I described the process in detail here. After baseline setup we are ready to go. To simulate delay I used this:

/* x is number of microseconds, for example usleep(10000)
blocks current thread for 10000 microseconds
or 0.01 second and usleep(1000000) blocks for 1 second */

usleep(x)
Enter fullscreen mode Exit fullscreen mode

I did not notice any difference between different launches in contrast to measuring scroll performance so there is only one try for each delay value. Following hitch-related metrics are not relevant in this case and during testing value for them is always zero:

  • Hitch Time Ratio (NavigationTransition): 0.000 ms per s
  • Hitches Total Duration (NavigationTransition): 0.000 ms
  • Number of Hitches (NavigationTransition): 0.000 hitches

I ignored these values during testing. For delay values I tried to find the pivot point when tests start to fail. Following table demonstrates test results:

Delay(ms) Simulator output Real device output
10 Passed: Duration: 0.527 s (2% worse) Passed: Duration: 0.532 s, Frame Count: 48.000 frames (2% better), Frame Rate: 90.293 fps (1% worse)
25 Passed: Duration: 0.542 s (5% worse) Passed: Duration: 0.549 s (6% worse), Frame Count: 48.600 frames (3% better), Frame Rate: 88.560 fps (3% worse)
50 Failed: Duration average is 11% worse (max allowed: 10%) Failed: Duration average is 12% worse (max allowed: 10%), Frame Count: 50.000 frames (6% better) Frame Rate: 86.641 fps (5% worse)
75 Failed: Duration average is 16% worse (max allowed: 10%) Failed: Duration average is 17% worse (max allowed: 10%).Frame Count: 52.000 frames (11% better), Frame Rate: 85.974 fps (5% worse)
100 Failed:Duration average is 22% worse (max allowed: 10%) Failed: Duration average is 23% worse (max allowed: 10%), Frame Count: 53.800 frames (14% better), Frame Rate: 84.672 fps (7% worse)

Results show the following:

  1. Duration is the most sensitive metric during measuring navigation performance.
  2. Simulator results are very similar to real device results.
  3. Tests start to fail both for a simulator and a real device with a delay equal to or more than 50 ms
  4. Frame Count and Frame Rate are not predictable in the described case and hardly usable

Conclusion

The described method is an efficient tool to detect navigation performance issues in early development stages. It has some constraints such as using the same device model for test recording and testing, but even with a simulator it can detect delays that are not noticeable for a human. In my opinion, this can be a great way of controlling crucial flows performance in your app.

Useful links

Writing and running performance tests

Image of Timescale

🚀 pgai Vectorizer: SQLAlchemy and LiteLLM Make Vector Search Simple

We built pgai Vectorizer to simplify embedding management for AI applications—without needing a separate database or complex infrastructure. Since launch, developers have created over 3,000 vectorizers on Timescale Cloud, with many more self-hosted.

Read more

Top comments (3)

Collapse
 
100rabh_kapoor profile image
Saurabh Kapoor

I have been working on writing perf tests for some time now and one issue that I frequently come across is that the results are not consistent. I have tried XCTOSSignpostMetrics for checking hitch ratio and also XCTMemoryMetric for checking the leaks.

Do you have any idea why there is inconsistency and also if we can do anything to have more consistent results across runs?

Collapse
 
mtmorozov profile image
Dmitrii Morozov

Have you tried using real device for testing?

Collapse
 
100rabh_kapoor profile image
Saurabh Kapoor

Yes, tried it on real devices but still see inconsistency in the results. I have also raised this issue with apple in the perf lab discussion earlier this year.