Measurements with MetricKit

Intro

Sometimes you need to measure something in the app. It can be different things like app launch time or just custom time spent between certain points of execution. One of the most convenient tools in this regard is MetricKit framework. With its help, you can measure different performance and diagnostic metrics, monitor regressions and identify problems in your app. The article describes the basic setup, output data format and possible options to process it. Also, it describes custom measurements and events reporting with MetricKit.

Getting started

Let’s start with a simple example: how to measure your app’s launch time?

Setup

First, we need to define an object responsible for handling info from the framework and implement MXMetricManagerSubscriber. This protocol contains two optional methods:
optional func didReceive(_ payloads: [MXMetricPayload])
optional func didReceive(_ payloads: [MXDiagnosticPayload])
In this article we will focus only on the first one that provides metrics.
According to docs:

The system calls this method at most once per day. It’s safe to process the payload on a separate thread.

In real life you can open an app every day but the callback will be called once in three days, it only guarantees to call the method no more than once a day. in my experience, it is called on average once every 2-3 days.
Let’s create a separate class responsible for metrics reporting:

import MetricKit

final class MetricReporter: MXMetricManagerSubscriber {
    func didReceive(_ payloads: [MXMetricPayload]) {
        // TODO: Report metrics
    }
}

Those methods are called in the background so they don’t affect app performance. Later I will explain how to parse data from those metrics. Now we need to have this object during the whole app lifecycle so let’s add it to AppDelegate.swift:

@main
class AppDelegate: UIResponder, UIApplicationDelegate {
    private lazy var metricReporter = MetricReporter()
    func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
        MXMetricManager.shared.add(metricReporter)
        return true
    }
}

So now we are ready to try payload example.

Payload overview

To get a sample for MetricKit we can use Xcode Debug -> Simulate MetricKit Payloads, but it is important to use a real device, for a simulator this option is disabled.

Every MXMetricPayload contains different instances of MXMetric subclasses such as MXAppLaunchMetric, MXAppExitMetric, MXMemoryMetric and others. All of them in turn contain different sets of following classes and structures:

Measurement structure represents value including information about units. This is how it looks like in the example of peak memory usage:

"peakMemoryUsage" : "200000 kB"

MXHistogram class represents the number of times the measured value falls into a specific range of possible values. This is how it looks in json form in the example of the launch time of the app:

"histogrammedTimeToFirstDrawKey" : {
      "histogramNumBuckets" : 3,
      "histogramValue" : {
        "0" : {
          "bucketEnd" : "1010 ms",
          "bucketCount" : 50,
          "bucketStart" : "1000 ms"
        },
        "1" : {
          "bucketEnd" : "2010 ms",
          "bucketCount" : 60,
          "bucketStart" : "2000 ms"
        },
        "2" : {
          "bucketEnd" : "3010 ms",
          "bucketCount" : 30,
          "bucketStart" : "3000 ms"
        }
      }
    }

MXAverage class represents average value including information about sample count and standard deviation. This is how it looks like in the example of average pixel luminance:

"averagePixelLuminance" : {
    "averageValue" : "50 apl",
    "standardDeviation" : 0,
    "sampleCount" : 500
  }

Processing

There are two different approaches for metrics data processing: preprocess data on a device and send it or send data as it is. Both have disadvantages and advantages, let’s take a look at the details.

Processing on device

Process data on mobile and send result values to your analytics solution. In general, you need to transform metrics-specific data structures into simpler ones, for example extract average value from MXHistogram data structure. A great example of this approach can be found here. Just a few important things to remember:

• Remember to check includesMultipleApplicationVersions field to filter out reports containing data from different versions
• Remember to filter out abnormal values with isNan , isNormal or isFinite checks. In the previously mentioned example author used isNan but you can also utilise isFinite or isNormal, but remember that zero is not a normal number

In this case, you select on mobile the exact data you want to send for analyzing. Unfortunately, if you make any mistake on this step it would be quite hard to catch since you don’t have access to raw data.

Offload processing computation off mobile devices

Offload processing computation off mobile devices and send data as it is for processing. In this great example the author uses a web service to process metric payloads. The main advantages of this method are:

1. All raw data is available so if later you discover any issues with your processing you can fix it without any consequences
2. At any point in time you can access data from the past

The main disadvantage here is the higher complexity.

Custom metrics

Apart from predefined measurements MetricKit also supports custom measurements and event tracking with mxSignpost

Custom measurements

This is how you can do it on the example of applying some heavy operation:

func apply() {
    // create log handler
    let handle = MXMetricManager.makeLogHandle(category: "ApplyCategory")
    mxSignpost(.begin, log: handle, name: "ApplyTrace")
    // critical code section begins
    // ...
    // critical code section ends
    // end measuring
    mxSignpost(.end, log: handle, name: "ApplyTrace")
}

func cancel() {
    let handle = MXMetricManager.makeLogHandle(category: "CancelCategory")
    mxSignpost(.begin, log: handle, name: "CancelTrace")
    // ...
    mxSignpost(.end, log: handle, name: "CancelTrace")
}

Though it looks pretty simple there are several important things to highlight:

• It is important to specify the same name in mxSignpost calls, otherwise, you will not get results in a report
• There is special note about this API usage:

The system limits the number of custom signpost metrics saved to the log in order to reduce on-device memory overhead. Limit the use of custom metrics to critical sections of code.

As a result, you get something similar to the following piece in result payloads:

{
  "signpostMetrics" : [
    {
      "signpostIntervalData" : {
        "histogrammedSignpostDurations" : {
          "histogramNumBuckets" : 2,
          "histogramValue" : {
            "0" : {
              "bucketCount" : 1,
              "bucketStart" : "0 ms",
              "bucketEnd" : "99 ms"
            },
            "1" : {
              "bucketCount" : 1,
              "bucketStart" : "100 ms",
              "bucketEnd" : "199 ms"
            }
          }
        },
        "signpostCumulativeCPUTime" : "262 ms",
        "signpostAverageMemory" : "64433 kB",
        "signpostCumulativeLogicalWrites" : "748 kB"
      },
      "signpostCategory" : "ApplyCategory",
      "signpostName" : "ApplyTrace",
      "totalSignpostCount" : 2
    },
    {
      "signpostIntervalData" : {
        "histogrammedSignpostDurations" : {
          "histogramNumBuckets" : 1,
          "histogramValue" : {
            "0" : {
              "bucketCount" : 81,
              "bucketStart" : "0 ms",
              "bucketEnd" : "99 ms"
            }
          }
        },
        "signpostCumulativeCPUTime" : "295 ms",
        "signpostAverageMemory" : "211037 kB",
        "signpostCumulativeLogicalWrites" : "168 kB"
      },
      "signpostCategory" : "CancelCategory",
      "signpostName" : "CancelTrace",
      "totalSignpostCount" : 81
    }
  ]
}

As you can see in the attached payload there is histogram data that allows you to collect information about the execution time for specific code sections.

Events tracking

Apart from begin and end, you can use event OSSignpostType and in this case in a payload you will receive a number of times this event occurred. Here is an example:

let handle = MXMetricManager.makeLogHandle(category: "TestViewController")

override func viewDidLoad() {
    super.viewDidLoad()
    mxSignpost(.event, log: handle, name: "viewDidLoad")
}

override func viewWillAppear() {
    super.viewWillAppear()
    mxSignpost(.event, log: handle, name: "viewWillAppear")
}

With this code we can get the following payload part:

"signpostMetrics" : [
    {
      "signpostCategory" : "TestViewController",
      "totalSignpostCount" : 5,
      "signpostName" : "viewDidLoad"
    },
    {
      "signpostCategory" : "TestViewController",
      "totalSignpostCount" : 5,
      "signpostName" : "viewWillAppear"
    }
  ]

As you can see this can be utilised as an event-collecting tool to build various types of event funnels

Conclusion

MetricKit provides wide options of reporting tools to monitor:

1. Predefined metrics like application launch time, network usage, memory usage, etc.
2. Custom metrics recorded with mxSignpost for critical code sections
3. Events tracking with mxSignpost for critical events

These options can be utilised in various scenarios such as performance related improvements or event funnels.