Performance Regression Testing for React Native

Introduction

Let's talk about developing React Native apps. Or, to be more precise about developing performant React Native apps.

React Native is an excellent framework for developing cross-platform mobile apps. It's fast; it's easy to use. However, certain things should be kept in mind while developing React Native apps. One of them is performance.

As React Native applications grow in size, their performance tends to deteriorate. React Native is a JavaScript framework, and JavaScript is a single-threaded language. While the UI is being rendered, the JavaScript thread is blocked, and no other JavaScript code can be executed.

React Native Re-renders

Re-renders are one of the most common causes of performance problems in React Native apps. They can go unnoticed in React web apps; however, in React Native apps, an excessive amount of re-renders leads to serious performance issues and slow UI. On top of that, it's pretty hard to spot code changes that lead to re-renders just by reviewing the code.

When a component re-renders, a serialized message is sent through the bridge from the JavaScript thread to the native platform. The native platform unserializes the message and creates a new native view, which is then rendered on the screen. If the render result is the same as the previous result, it may not result in native view changes; however, the time it takes to re-render a portion of the element tree might block the JS thread.

If the JavaScript thread is unresponsive for a frame, it will be considered a dropped frame. Any animations controlled by JS would appear to freeze during that time; and if it lasts longer than 100ms, the user will surely feel it. Not only animations are impacted; response to touches can be affected as well. In fact, there are quite a lot of performance issues that re-renders can cause. You can read about possible causes of such problems in the following guide on performance for possible causes of performance problems
https://www.callstack.com/campaigns/download-the-ultimate-guide-to-react-native-optimization

Development lifecycle

Development lifecycle would make a good topic for a separate blog post, but let me be brief here. In a nutshell, a simplified version of what happens during the development process is as follows:

1. New feature is introduced
2. Tests are added
3. QA goes on
4. Release takes place

Looks good, right? Especially because in reality, tests are sometimes skipped, or the QA process is rushed for the sake of releasing to prod. Let's assume a best-case scenario where test coverage is maintained in the high percentile, coverage reports are monitored, integration tests are automated, regression tests are executed, etc.

Even in this perfect setting, there might be bugs that escape the QA engineer's notice. But more importantly, performance problems that fail to be recognized by any test or QA process can occur.

Adding additional re-render can sound like no big deal, but performance degradation is like a snowball. As more features are developed, it gets more excruciating to pinpoint or fix performance issues. And once they become apparent to end users, it usually requires a lot of refactoring and a dedicated team working for hours to find the problem and carry out improvements. These, in turn, can introduce breaking changes to the rest of the app and lead to months of further development work.

Introducing Performance Regression testing

So, what can we do about this snowball of performance degradation growing over time? How can we know that a new feature reduces performance? How can we catch the unnecessary re-renders and get actual performance metrics without the overhead of having a dedicated team working on performance improvement?

Of course, there is no silver bullet to finding ALL performance problems, especially the ones that originate on the native side. Still, we can address the ones that arise from incorrect usage of React or React Native, like long lists, not using memoization, unnecessary re-renders, etc.

It's not enough to only identify these performance issues; we also need to introduce some measuring tools into development workflow and CI to automate performance regression tests. While doing it manually on each PR review is possible, it takes precious development time, introduces complexity to the release process, and increases the chance of performance problems slipping through the PR review process and ending up in prod.

Reassure for the rescue

Given that there hasn't been any tool for automating performance regression testing so far, folks at Callstack have decided to create one: https://www.callstack.com/open-source/reassure

Introducing a new tool means introducing a new workflow. This can lead to lower adoption, as developers need to to get familiar with it in the first place. To prevent that from happening, we've based Reassure on https://callstack.github.io/react-native-testing-library/ API as something React Native developers should be familiar with.

Reassure can be easily integrated with your current React Native Testing Library setup. Writing performance tests is pretty close to writing regular tests, as we will see in examples later in this blog post.

Adding Reassure to your project

yarn add --dev reassure

or

npm install --save-dev reassure

After you've added reassure you can start writing performance regression tests by adding .perf-test.tsx extension.

Since there is a similarity between your regular tests and reassure performance regression tests, the process of creating the latter will usually involve copy-pasting existing tests and altering them to match reassure API.

In a nutshell, instead of using render function from React Native Testing library, you would use measurePerfomance function and pass the scenario function to it, which will execute the scenario to measure.

To run reassure, you will run yarn reassure and get the results.

Let's take a look at a more detailed example, so we can understand how we can not only measure the performance of an existing component, but also refactor it while significantly improving its performance.

Refactor long lists while measuring performance

Consider the following GameList component.

export const GameList = () => {
  const {isLoading, data} = useGetGames();

  const [gameLikes, setGameLikes] = React.useState<any>({});

  const likeGame = (id: number) => {
    setGameLikes((prevState: any) => {
      return {
        ...prevState,
        [id]: prevState[id] ? prevState[id] + 1 : 1,
      };
    });
  };

  return isLoading ? (
    <Box
      flex={1}
      backgroundColor="black"
      alignItems="center"
      justifyContent="center">
      <Spinner color="purple.500" size="lg" testID="spinner" />
    </Box>
  ) : (
    <View testID="list">
      <Box>
        {data?.pages
          ?.map((page: {results: any}) => page.results)
          .flat()
          .map((game: any) => (
            <GameItem
              game={game}
              likes={gameLikes[game.id]}
              likeFn={() => likeGame(game.id)}
            />
          ))}
      </Box>
    </View>
  );
};

This component will get games data from useGames hook and render it on the screen. Before measuring component performance and optimizing it, let's first write a simple test for it.

We will create GameList.test.tsx file and write our test there.

First, we need to mock useGames hook:

jest.mock('../hooks', () => ({
  useGetGames: () => ({
    data: {
      pages: [
        {
          results: Array.from({length: 1000}, () => ({
            id: Math.random() * 10000,
            name: Math.random().toString(36).substring(7),
            background_url: `https://i.imgur.com/${Math.random()
              .toString(36)
              .substring(2)}.jpg`,
          })),
        },
      ],
    },
    hasNextPage: true,
    fetchNextPage: jest.fn(),
    isFetchingNextPage: false,
  }),
}));

Initially, our component is pulling data from https://api.rawg.io/api, so we will need to match results to be in the same structure as returned from API. As you can see in the code above, we will generate 1000 game items with random ids, random names, and background_url pulled from imgur

Note: For the sake of simplicity in this particular example, we use jest.mock to mock our API results. Yet, it's advised to use msw to mock Api call results, which you can read more about here

Now, let's write the test.

const initialWindowMetrics = {
    frame: {
      x: 0,
      y: 0,
      width: 0,
      height: 0,
    },
    insets: {
      top: 0,
      left: 0,
      right: 0,
      bottom: 0,
    },
  };

test('Renders GameList', async () => {
  const createWrapper = () => {
    return ({children}) => (
      <NativeBaseProvider initialWindowMetrics={initialWindowMetrics}>
        {children}
      </NativeBaseProvider>
    );
  };
  const wrapper = createWrapper();
  render(<GameList />, {wrapper});
  const gameItems = await screen.findAllByTestId('game-item');
  expect(gameItems).toHaveLength(1000);
  expect(screen.queryByTestId('spinner')).not.toBeTruthy();
  expect(screen.queryByTestId('list')).toBeTruthy();
});

We are using NativeBase for our UI in this test, which means we also have to wrap our GameList with NativeBaseProvider and supply it with initialWindowMetrics, so it can successfully render our UI.

In this test, we have 1000 game items and list component is present.The test passes, but we can probably identify the problem here. There are 1000 rendered game items. Clearly, our component is not optimized and will render all 1000 items, even though they are not visible to the user initially.

Measuring performance with Reassure

In the previous section, we've set up Reassure, so now, let's use it to measure the performance of our component.

Let's copy our GameList.test.tsx to be GameList.perf-test.tsx and alter our performance test to match Reassure API.

Mocking

We will reuse the same mock we've created in GameList.test.tsx, but since the performance test will take way more time to run, we need to limit the test to only 100 items instead of 1000 and increase jest default timeout from 5000ms to a minute by using jest.setTimeout(60_000);

Note that to get a more reliable performance measurement, Reassure executes each scenario ten times and averages the results (the number of runs can be easily configured using configure function

Due to a slight API difference between Reassure and React Native Testing library, we also need to change our createWrapper function from the previous test in the following way:

const createWrapper = () => {
  return (node: JSX.Element) => (
    <NativeBaseProvider initialWindowMetrics={initialWindowMetrics}>
      {node}
    </NativeBaseProvider>
  );
};

Then we eliminate all our assertions and call measurePerformance function from Reassure, passing it to our wrapper.

The final performance test looks like this:

import React from 'react';
import {NativeBaseProvider} from 'native-base';
import {measurePerformance} from 'reassure';
import {GameList} from './GameList';
jest.setTimeout(60_000);
jest.mock('../hooks', () => ({
  useGetGames: () => ({
    data: {
      pages: [
        {
          results: Array.from({length: 100}, () => ({
            id: Math.random() * 10000,
            name: Math.random().toString(36).substring(7),
            background_url: `https://i.imgur.com/${Math.random()
              .toString(36)
              .substring(2)}.jpg`,
          })),
        },
      ],
    },
    hasNextPage: true,
    fetchNextPage: jest.fn(),
    isFetchingNextPage: false,
  }),
}));

test('GameList perf test', async () => {
  const initialWindowMetrics = {
    frame: {
      x: 0,
      y: 0,
      width: 0,
      height: 0,
    },
    insets: {
      top: 0,
      left: 0,
      right: 0,
      bottom: 0,
    },
  };

  const createWrapper = () => {
    return (node: JSX.Element) => (
      <NativeBaseProvider initialWindowMetrics={initialWindowMetrics}>
        {node}
      </NativeBaseProvider>
    );
  };

  const wrapper = createWrapper();

  await measurePerformance(<GameList />, {
    wrapper,
  });
});

At this point, we can run yarn reassure --baseline to gather baseline metrics to compare during all subsequent runs.

After running this command, .reassure/baseline.perf file is created with the following data:

{"metadata":{}}
{"name":"GameList perf test","runs":10,"meanDuration":480.2,"stdevDuration":31.957610813214572,"durations":[516,506,505,504,495,488,473,456,431,428],"meanCount":1,"stdevCount":0,"counts":[1,1,1,1,1,1,1,1,1,1]}

Since there can be discrepancies between the measured performance metrics, Reassure is designed to execute multiple runs (10 by default) and provide the average for two main parameters:

• render count
• render duration

In the .perf file, we can see the exact rendering duration and the number of renders for each run. We can increase the number of runs by providing runs: number option to measurePerfomance function.

Introducing scenarios

While measuring performance, we usually want to check whether the component is re-rendered due to UI interaction. In our example, when clicking on a game item, likes for this item are updated. The way it works at the moment is by setting the state on the component and, as a result, triggering GameList re-render.

Let's validate this scenario in our performance test and see how it will impact our metrics.

We create a scenario async function where we can use any React Native Testing library methods.

const scenario = async () => {
  const gameItems = await screen.findAllByTestId('game-item');
  fireEvent(gameItems[0], 'press');
};

Now we can pass this scenario to measurePerformance function:

await measurePerformance(<GameList />, {
  scenario,
  wrapper,
});

Now, if we run yarn reassure, Reassure will compare .reassure/baseline.perf file with .reassure/current.perf file, and we will see the following results:

✅ Written Current performance measurements to .reassure/current.perf
🔗 /Users/vladimirnovick/dev/Callstack/Reassure/reassuredemo/.reassure/current.perf

❇️ Performance comparison results:

• Current: (unknown)
• Baseline: (unknown)

➡️ Signficant changes to render duration

• GameList perf test: 480.2 ms → 690.7 ms (+210.5 ms, +43.8%) 🔴🔴 | 1 → 2 >(+1, +100.0%) 🔴

➡️ Meaningless changes to render duration

➡️ Render count changes

• GameList perf test: 480.2 ms → 690.7 ms (+210.5 ms, +43.8%) 🔴🔴 | 1 → 2 >(+1, +100.0%) 🔴

➡️ Added scenarios

➡️ Removed scenarios

Here we can see an increase in the render count of the component and 43.8% increase in rendering time.

It's an excellent showcase of how new interactions during the development of new features can gradually increase rendering time and decrease the component's performance. Having a tool like Reassure gives us a detailed insight into how components' performance changes as the app grows.

Improving performance

Let's set this new metric as our new baseline and start improving our component to be more performant, measuring the number of re-renders and rendering time as we do our refactoring.

First of all, to set up a new baseline, let's run yarn reassure --baseline.

As mentioned before, it will write new metrics in baseline.perf file and use it for comparison later on

Refactoring to FlatList

You might be wondering, why not use FlatList? That’s a fair question. We shouldn't map over items, but rather use FlatList to render game items on the screen. So let's refactor our component accordingly:

<FlatList
  data={data?.pages?.map((page: {results: any}) => page.results).flat()}
  renderItem={renderData}
  keyExtractor={gameItemExtractorKey}
  onEndReached={loadMore}
  onEndReachedThreshold={0.3}
  ListFooterComponent={isFetchingNextPage ? renderSpinner : null}
/>

Results will be outstanding:

Written Current performance measurements to .reassure/current.perf
🔗 /Users/vladimirnovick/dev/Callstack/Reassure/reassuredemo/.reassure/current.perf

❇️ Performance comparison results:

• Current: (unknown)
• Baseline: (unknown)

➡️ Signficant changes to render duration

• GameList perf test: 687.6 ms → 83.8 ms (-603.8 ms, -87.8%) 🟢🟢 | 2 → 2

➡️ Meaningless changes to render duration

➡️ Render count changes

➡️ Added scenarios

➡️ Removed scenarios

As you can see, there is an 87.8% improvement in rendering times.

If you run yarn test GameList.test.tsx to check if our unit test is working, it will fail due to the FlatList rendering only 10 game items out of 1000 items available. That’s because the remaining items are off-screen and do not need any rendering.

Let's also change our GameList.test.tsx test and switch from

expect(gameItems).toHaveLength(1000);

to

expect(gameItems).toHaveLength(10);

For further improvement, let's move likes logic into GameItem component. This will yield the following results:

Performance comparison results:

• Current: (unknown)
• Baseline: (unknown)

➡️ Signficant changes to render duration

• GameList perf test: 687.6 ms → 47.3 ms (-640.3 ms, -93.1%) 🟢🟢 | 2 → 2

➡️ Meaningless changes to render duration

➡️ Render count changes

➡️ Added scenarios

➡️ Removed scenarios

✅ Written JSON output file .reassure/output.json
🔗 /Users/vladimirnovick/dev/Callstack/Reassure/reassuredemo/.reassure/output.> json

✅ Written output markdown output file .reassure/output.md
🔗 /Users/vladimirnovick/dev/Callstack/Reassure/reassuredemo/.reassure/output.md

As you can see from this example, we've successfully improved the performance of our component by 91.4% and reduced rendering times from 687.6ms to 59.3ms.

Using Reassure for performance regression testing in CI

Even though in our example, we've used Reassure as a tool to improve and measure performance, the area where the tool shines the brightest is Performance Regression testing.

As we develop new features, it's easy to gradually degrade performance until the problem becomes noticeable to the user. With Reassure, you can compile a report of performance changes by running Reassure in CI on your PRs.

It's important to note that the speed of the machine on which Reassure is running matters. To resolve any potential inconsistencies in the results, Reassure runs tests multiple times, averages the results, and provides a handy CLI tool to assess machine stability.

By running yarn reassure check-stability, it carries out performance measurements for the current code twice, so that baseline and current measurements refer to the same code. In such a case, the expected changes are 0% (no change). The degree of random performance changes will reflect the stability of your machine. This command can be run both on CI and local machines.

When integrating Reassure into CI, your CI agent stability is even more important than its speed. That’s because we want to see consistent results.

Tip: If your machine or CI agent stability is so-so, consider increasing the number of runs in the Reassure configure function

You can create a custom script for CI to run to compare performance between your main branch and the current branch

#!/usr/bin/env bash
set -e

BASELINE_BRANCH=${BASELINE_BRANCH:="main"}

# Required for `git switch` on CI
git fetch origin

# Gather baseline perf measurements
git switch "$BASELINE_BRANCH"
yarn install --force
yarn reassure --baseline

# Gather current perf measurements & compare results
git switch --detach -
yarn install --force
yarn reassure

In the script above, we set the baseline branch to be the main branch and run yarn reassure --baseline to gather baseline metrics on it.

Then we compare them with the current branch by running yarn reassure.

You can read more about setting up Reassure in CI in official docs here

In this article, I presented a few simple examples of things that can cause performance problems in the long run. As you've noticed, minor refactoring led to a 93.1% performance improvement. Even though performance issues presented in the article can be caught during a thorough PR review, that can become inherently harder in complex codebases and as your React Native app grows. Making Reassure part of your test suite to address potential performance regressions and automate things in CI would greatly benefit you in the long run. Also, if you already write tests with react-native-testing-library (If you don't, you definitely should), creating Reassure perf tests is, to some degree, a copy-paste of existing tests with slight modification so that it can be quickly introduced into the development workflow.
Due to the open-source nature of Reassure, if you have improvement suggestions, feel free to create issues in the repo, and Reassure team will address them.