Tap Response Time: Jetpack Navigation 🗺

Header image: Surf by Romain Guy.

In Android Vitals - Tap Response Time 👉 we established that the naive approach to measuring Tap Response Time isn't accurate and doesn't scale. Today we'll build a better implementation step by step, on top Jetpack Navigation.

🗺 Navigation library

We'll focus on Jetpack Navigation here, however most of the content applies for any navigation library or tap action. In fact, I first implemented this at Square on top of Flow and Workflow.

Advanced Navigation Sample

We'll implement the Tap Response Time measurement inside the Advanced Navigation Sample and focus on the navigation from the Title screen to the About screen.

aboutButton.setOnClickListener {
  findNavController().navigate(R.id.action_title_to_about)
}

From Tap to Render

What happens exactly when we click on the about button?

Main thread tracing

To figure that out, we enable Java method tracing while clicking on the button :

1. The MotionEvent.ACTION_UP event is dispatched and a click is posted to the main thread.
2. The posted click runs, the click listener calls NavController.navigate() and a fragment transaction is posted to the main thread.
3. The fragment transaction runs, the view hierarchy is updated, and a view traversal is scheduled for the next frame on the main thread.
4. The view traversal runs, the view hierarchy is measured, laid out and drawn.

What happens after step 4?

Systrace

We get a better high level view with systrace:

• In step 4, the view traversal draw pass generates a list of drawing commands (known as display lists) and sends that list of drawing commands to the render thread.
• Step 5: the render thread optimizes the display lists, adds effects such as ripples, then leverages the GPU to run the drawing commands and draw into a buffer (an OpenGL surface). Once done, the render thread tells the surface flinger (which lives in a separate process) to swap the buffer and put it on the display.
• Step 6 (not visible in the systrace screenshot): the surfaces for all visible windows are composited by the surface flinger and hardware composer, and the result is sent to the display.

Tap Response Time

We previously defined the Tap Response Time as the time from when the user is done pressing a button to when the app has visibly reacted to the tap. In other words, we need to measure the total duration of going through steps 1 to 6.

In the next sections I'll explain how we can detect each step. If you'd rather immediately go to the final implementation, here's the PR against NavigationAdvancedSample.

Step 1: Up dispatch

We leverage square/curtains to intercept touch events. We define TapTracker, a touch event interceptor. TapTracker stores the time of the last MotionEvent.ACTION_UP touch event. When the posted click listener triggers, we retrieve the time of the up event that triggered it by calling TapTracker.currentTap:

object TapTracker : TouchEventInterceptor {

  var currentTap: TapResponseTime.Builder? = null
    private set

  private val handler = Handler(Looper.getMainLooper())

  override fun intercept(
    motionEvent: MotionEvent,
    dispatch: (MotionEvent) -> DispatchState
  ): DispatchState {
    val isActionUp = motionEvent.action == MotionEvent.ACTION_UP
    if (isActionUp) {
      val tapUptimeMillis = motionEvent.eventTime
      // Set currentTap right before the click listener fires
      handler.post {
        TapTracker.currentTap = TapResponseTime.Builder(
          tapUptimeMillis = tapUptimeMillis
        )
      }
    }
    // Dispatching posts the click listener.
    val dispatchState = dispatch(motionEvent)

    if (isActionUp) {
      // Clear currentTap right after the click listener fires
      handler.post {
        currentTap = null
      }
    }
    return dispatchState
  }
}

We then add the TapTracker interceptor to each new window:

class ExampleApplication : Application() {

  override fun onCreate() {
    super.onCreate()

    Curtains.onRootViewsChangedListeners +=
      OnRootViewAddedListener { view ->
        view.phoneWindow?.let { window ->
          if (view.windowAttachCount == 0) {
            window.touchEventInterceptors += TapTracker
          }
        }
      }
  }
}

Step 2: Click listener & navigation

Let's define an ActionTracker that will be called when when the posted click listener triggers:

object ActionTracker {
  fun reportTapAction(actionName: String) {
    val currentTap = TapTracker.currentTap
    if (currentTap != null) {
      // to be continued...
    }
  }
} 

Here's how we could leverage it:

aboutButton.setOnClickListener {
  findNavController().navigate(R.id.action_title_to_about)
  ActionTracker.reportTapAction("About")
}

However, we don't want to add that code to every click listener. Instead, we can add a destination listener to the NavController:

navController.addOnDestinationChangedListener { _, dest, _ ->
  ActionTracker.reportTapAction(dest.label.toString())
}

The Advanced Navigation Sample has 3 tabs, and each tab contains its own NavHostFragment and NavController. We could add a destination listener for each tab. Or we can leverage lifecycle callbacks to add a destination listener to every new NavHostFragment instance:

class GlobalNavHostDestinationChangedListener
  : ActivityLifecycleCallbacks {

  override fun onActivityCreated(
    activity: Activity,
    savedInstanceState: Bundle?
  ) {
    if (activity is FragmentActivity) {
      registerFragmentCreation(activity)
    }
  }

  private fun registerFragmentCreation(activity: FragmentActivity) {
    val fm = activity.supportFragmentManager
    fm.registerFragmentLifecycleCallbacks(
      object : FragmentLifecycleCallbacks() {
        override fun onFragmentCreated(
          fm: FragmentManager,
          fragment: Fragment,
          savedInstanceState: Bundle?
        ) {
          if (fragment is NavHostFragment) {
            registerDestinationChange(fragment)
          }
        }
      }, true
    )
  }

  private fun registerDestinationChange(fragment: NavHostFragment) {
    val navController = fragment.navController
    navController.addOnDestinationChangedListener { _, dest, _ ->
      val actionName = dest.label.toString()
      ActionTracker.reportTapAction(actionName)
    }
  }
}

Step 3: Fragment transaction

Calling NavController.navigate() does not immediately update the view hierarchy. Instead, a fragment transaction is posted to the main thread. The view for the destination fragment will be created and attached when the fragment transaction executes. Since all pending fragment transactions are executed at once, we add our own custom transaction to leverage the runOnCommit() callback. Let's first build a utility, OnTxCommitFragmentViewUpdateRunner.runOnViewsUpdated():

class OnTxCommitFragmentViewUpdateRunner(
  private val fragment: Fragment
) {
  fun runOnViewsUpdated(block: (View) -> Unit) {
    val fm = fragment.parentFragmentManager
    val transaction = fm.beginTransaction()
    transaction.runOnCommit {
      block(fragment.view!!)
    }.commit()
  }
}

We then pass an instance to ActionTracker.reportTapAction():

class GlobalNavHostDestinationChangedListener
 ...
     val navController = fragment.navController
     navController.addOnDestinationChangedListener { _, dest, _ ->
       val actionName = dest.label.toString()
-      ActionTracker.reportTapAction(actionName)
+      ActionTracker.reportTapAction(
+        actionName,
+        OnTxCommitFragmentViewUpdateRunner(fragment)
+      )
     }
   }
 }

 object ActionTracker {
-  fun reportTapAction(actionName: String) {
+  fun reportTapAction(
+      actionName: String,
+      viewUpdateRunner: OnTxCommitFragmentViewUpdateRunner
+  ) {
     val currentTap = TapTracker.currentTap
     if (currentTap != null) {
-      // to be continued...
+      viewUpdateRunner.runOnViewsUpdated { view ->
+        // to be continued...
+      }
     }
   }
 }

Step 4: Frame & view hierarchy traversal

When the fragment transaction executes, a view traversal is scheduled for the next frame, which we hook into with Choreographer.postFrameCallback():

 object ActionTracker {
+
+  // Debounce multiple calls until the next frame
+  private var actionInFlight: Boolean = false
+
   fun reportTapAction(
       actionName: String,
       viewUpdateRunner: OnTxCommitFragmentViewUpdateRunner
   ) {
     val currentTap = TapTracker.currentTap
-    if (currentTap != null) {
+    if (!actionInFlight & currentTap != null) {
+      actionInFlight = true
       viewUpdateRunner.runOnViewsUpdated { view ->
-        // to be continued...
+        val choreographer = Choreographer.getInstance()
+        choreographer.postFrameCallback { frameTimeNanos ->
+          actionInFlight = false
+          // to be continued...
+        }
       }
     }
   }
 }

Step 5: RenderThread

Once the view traversal is done, the main thread sends the display lists to the render thread. The render thread does additional work and then tells the surface flinger to swap the buffer and put it on the display. We register a OnFrameMetricsAvailableListener to get the total frame duration (including time spent on the render thread):

 object ActionTracker {
 ...
         val choreographer = Choreographer.getInstance()
         choreographer.postFrameCallback { frameTimeNanos ->
           actionInFlight = false
-          // to be continued...
+          val callback: (FrameMetrics) -> Unit = { frameMetrics ->
+            logTapResponseTime(currentTap, frameMetrics)
+          }
+          view.phoneWindow!!.addOnFrameMetricsAvailableListener(
+            CurrentFrameMetricsListener(frameTimeNanos, callback),
+            frameMetricsHandler
+          )
         }
       }
     }
   }
+
+  private fun logTapResponseTime(
+    currentTap: TapResponseTime.Builder,
+    fM: FrameMetrics
+  ) {
+    // to be continued...
+  }

Once we have the frame metrics we can determine when the frame buffer was swapped, and therefore the Tap Response Time, i.e. the time from MotionEvent.ACTION_UP to buffer swap:

 object ActionTracker {
 ...
     currentTap: TapResponseTime.Builder,
     fM: FrameMetrics
   ) {
-    // to be continued...
+    val tap = currentTap.tapUptimeMillis
+    val intendedVsync = fM.getMetric(INTENDED_VSYNC_TIMESTAMP)
+    // TOTAL_DURATION is the duration from the intended vsync
+    // time, not the actual vsync time.
+    val frameDuration = fM.getMetric(TOTAL_DURATION)
+    val bufferSwap = (intendedVsync + frameDuration) / 1_000_000
+    Log.d("TapResponseTime", "${bufferSwap-tap} ms")
   }
 }

Step 6: SurfaceFlinger

There's no Java API to determine when the composited frames end up being sent to the display by SurfaceFlinger, so I didn't include that part. Romain Guy mentioned this can be done with a native call to EGL_ANDROID_get_frame_timestamps.

Result

When we click on the About button, we now see a nice log in Logcat:

D/TapResponseTime: 105 ms

I ran systrace at the same time. As you can see in the screenshot, the time from tap to buffer swap is also 105ms:

Conclusion

You can see the final result in this PR. Feel free to leave comments on the PR! I added a few more things that I didn't cover in this blog, such as back key support and logging tab navigation.

I intend to eventually create a Square Open Source library for this. Until then, you have everything you need to get started.

My hope is that you can leverage this code to start measuring tap response time in your production apps, which will help you improve the experience of your customers 👏.

⚖️ This work is licensed under a Creative Commons Attribution 4.0 International License

Comments

[Tudor Luca]

I've been playing with Workflow & Compose lately. I'm curios to see how the numbers for Workflow + View.java look like, if you can share them.