A/B Testing with Create React App

Create React App (CRA) is an extremely popular framework for building front-end applications.

Legacy A/B testing tools like Optimizely, VWO, and Google Optimize were built during the jQuery days and fall flat in modern React applications. Those tools work by applying changes on top of the DOM (e.g. dynamically changing the src of an <img> tag). That's exactly how React works as well, which is where the problems start. These tools end up in an arms race where React will re-render a component and then the A/B testing script has to quickly switch it back before the user notices. This requires massive complexity and results in huge (100Kb+) javascript files that are slow and fragile.

GrowthBook is a new open source A/B testing platform built from the ground up for modern tech stacks. By leveraging the power of React (instead of fighting against it), GrowthBook is able to provide a really powerful and flexible A/B testing library in under 3Kb.

1. Set up Create React App

One of the best things about Create React App is how simple it is to get a new project started. We'll do the standard process with one extra command to install the GrowthBook SDK and the nanoid library:

npx create-react-app my-app
cd my-app
npm install --save @growthbook/growthbook-react nanoid
npm start

Then open http://localhost:3000/ and you're good to go.

2. Set up GrowthBook

In this tutorial, I'm going to use the cloud-hosted version of GrowthBook, which is free for small teams, but you can also use the open source version and host it yourself if that's more your style.

GrowthBook uses the concept of Feature Flagging to run A/B tests. Basically, you wrap the code you want to test in a conditional check if (feature.on) {...} and then you run an A/B test within GrowthBook to turn the feature on for 50% of users and off for the other 50%.

To start, go to https://app.growthbook.io and register a new account. Then there are a couple steps required before you can run an experiment.

3. Install and configure the SDK

Next, click on Step 1: Install our SDK and you should see API keys for dev/production as well as sample code.

We already ran the npm i command at the start, so you can skip that part. We can use the React code example as a starting point. I'll walk through the different parts below:

First, in src/index.js, we import the GrowthBook SDK and nanoid library:

// ... after existing imports
import { 
  GrowthBook, 
  GrowthBookProvider 
} from "@growthbook/growthbook-react";
import { nanoid } from "nanoid";

Then we need to generate an anonymous visitor id, which is used to assign an A/B test variation to a user. We'll persist this in localStorage so if the user refreshes our app they will get assigned the same variation as before.

let visitor_id = localStorage.get("visitor_id");
if(!visitor_id) {
  visitor_id = nanoid()
  localStorage.set("visitor_id", visitor_id);
}

Then, we create a GrowthBook instance with our visitor id and a tracking callback when a user is put into an experiment.

const growthbook = new GrowthBook({
  attributes: {
    id: visitor_id
  },
  trackingCallback: (experiment, result) => {
    console.log({
      experimentId: experiment.key, 
      variationId: result.variationId
    })
  }
});

After that, we fetch the list of features from the GrowthBook API and pass them into the SDK:

const FEATURES_ENDPOINT = 
  'https://cdn.growthbook.io/api/features/...';

fetch(FEATURES_ENDPOINT)
  .then((res) => res.json())
  .then((json) => {
    growthbook.setFeatures(json.features);
  });

Make sure to swap out the FEATURES_ENDPOINT constant above with your own dev API key you see in the GrowthBook application.

Lastly, we need to wrap the app in a GrowthBookProvider component which will let us run A/B tests from anywhere in the app.

ReactDOM.render(
  <React.StrictMode>
    <GrowthBookProvider growthbook={growthbook}>
      <App />
    </GrowthBookProvider>
  </React.StrictMode>,
  document.getElementById('root')
);

4. Create and use a feature

Now that the SDK is installed and fully integrated in our application, we can finally create the show-logo feature.

Back in GrowthBook, Click on Step 2: Add your first feature and fill in the following info:

• Feature key - show-logo
• Dev - toggle on
• Prod - toggle off
• Value Type - boolean (on/off)
• Behavior - A/B Experiment
• Tracking Key - show-logo
• Sample Users based on attribute - id
• Variations and Weights - leave default (OFF/ON, 50/50 split)
• Fallback Value - OFF

There's a lot of fields there, but hopefully it's pretty straight forward what's happening. We setup a new boolean feature called show-logo, that's only enabled in dev and running an A/B test where 50% get ON and 50% get OFF

Now we can switch back to our React app and reference this feature in our code.

In src/App.js, we currently have the following code:

<img src={logo} className="App-logo" alt="logo" />

Let's add an import at the top of the file:

import { IfFeatureEnabled } from '@growthbook/growthbook-react';

And wrap the img element in an IfFeatureEnabled component:

<IfFeatureEnabled feature="show-logo">
  <img src={logo} className="App-logo" alt="logo" />
</IfFeatureEnabled>

Now, if you refresh your app, the A/B test should be running! If you're part of the lucky 50% that are in the B variation (no logo), it should be pretty obvious. If you happen to be in the A variations, you can verify you're in the test by looking in DevTools for our trackingCallback console.log.

You can test out different variations by deleting the visitor_id from localStorage and refreshing your app. Repeat a few times and you should see each version of the page about half of the time. If you want an easier and faster way to QA the variations, you can download the GrowthBook Chrome DevTools Extension.

5. Analyze Results

Just running an A/B test by itself is fun, but not that useful. You also need to track and analyze the results.

In the trackingCallback in src/index.js, instead of doing a console.log, we could use Mixpanel or Segment or another event tracking system.

Then, throughout our app, we could similarly track events when users do something we care about, like sign up or buy something.

Once you do that, GrowthBook can connect to your event tracking system, query the raw data, run it through a stats engine, and show you the results. This process is a little more involved to set up, but I will walk through it in a followup post.

Next Steps

There's so much more you can do with GrowthBook beyond a simple on/off A/B test!

• Add complex targeting and override rules for your features
• Set up webhooks to keep a cached copy of features in your database
• Use the useFeature React hook for more control in your code
• Install the Chrome DevTools Extension to test different variations and scenarios
• Read about the powerful statistics engine that is used to analyze experiment results.

Comments

[Angella Naigaga]

Thanks for writing this tutorial. It came in handy after days of reading the documentation and not getting proper examples. Could you have an idea on how to include amplitude in the call backs?