Part 1: Create an A/B test with Nextjs, Vercel edge functions, and measure analytics with amplitude

You're getting tons of traffic to your website but conversions aren't great. You decide to run an A/B test to see if you can increase your pageview -> sign-up metrics. In this tutorial, we'll go over how to set up a simple A/B test with Nextjs, Vercel edge functions, and measure analytics with amplitude.

Part 1: Github Repo
Part 1: Site Example

Note: We will not get into good experimentation practices in this article. This tutorial is only to show how to effectively set up an A/B test using edge functions and measure with amplitude

Step 1: Create a new Nextjs app

npx create-next-app -e with-tailwindcss feedback-widget

Open up the new app in your code editor and we'll start building out our test!

Step 2: Setting up your experiment

Next, we'll need to set up the experiment. We decide to test button color (purple vs blue) to see if we can increase conversions. This is our first experiment so well name it exp001 and our experiment cohorts exp001-control (purple button) and exp001-variant (blue button).

Note: We choose button color to keep it simple for this test. Changing a color is generally not an acceptable A/B test to run.

Create an experiment folder in your project. Within the experiment folder, we'll need two files ab-testing.js and exp001.js.

Stetting up the cohorts

We have already decided on our two cohorts and their names for the experiment. These need to be set up as constants to use throughout the project. In your exp001.js file, we will name the cohorts and cookie:

// experiment cohort names
export const COHORTS = ['exp001-control', 'exp001-variant'];
// experiment cookie name
export const COOKIE_NAME = 'exp001-cohort';

Splitting traffic

Now that we have our cohorts, in our ab-testing file, we will set up our traffic splittitng. At the top of the file, create a function to generate a random number:

function cryptoRandom() {
  return (
    crypto.getRandomValues(new Uint32Array(1))[0] / (0xffffffff + 1)
  );
}

In our case we use crypto.getRandomValues() - you can always use Math.random() (we won't debate the differences between the two in this tutorial - follow good practice and use what you know best!). This function will give us a random number between 0 & 1. Next, create a function that names the cohort based on the random number above:

export function getCohort(cohorts) {
  // Get a random number between 0 and 1
  let n = cryptoRandom() * 100;
  // Get the percentage of each cohort
  const percentage = 100 / cohorts.length;
  // Loop through the cohors and see if the random number falls
  // within the range of the cohort
  return (
    cohorts.find(() => {
      n -= percentage;
      return n <= 0;
      // if error fallback to control
    }) ?? cohorts[0]
  );
}

The getCohorts() function above breaks the cohorts into an even split depending on the number of cohorts.

Now that we have our cohorts and traffic splitting function. We'll set up our homepage for the test.

Step 3: Middleware

What is middleware at the edge?

Vercel edge functions allow you to deploy middleware to the edge - close to your visitor's origin. Middleware is the actual code the runs before a request is processed. You can execute many different functions using middleware such as running an A/B test as we are here, blocking bots, and redirects just to name a few. The middleware function runs before any requests to your pages are completed.

Setting up our traffic splitting middleware

To run middleware we need to create a _middleware.js file in our pages directory. This middleware will run before any page request is completed.

import { getCohort } from '../experiment/ab-testing';
import { COHORTS, COOKIE_NAME } from '../experiment/exp001';

export function middleware(req) {
  // Get the cohort cookie
  const exp001 = req.cookies[COOKIE_NAME] || getCohort(COHORTS);
  const res = NextResponse.rewrite(`/${exp001}`);

  // For a real a/b test you'll want to set a cookie expiration
  // so visitors see the same experiment treatment each time
  // they visit your site

  // Add the cohort name to the cookie if its not there
  if (!req.cookies[COOKIE_NAME]) {
    res.cookie(COOKIE_NAME, exp001);
  }

  return res;
}

The middleware first attempts to get the cohort cookie if there is one and if not, runs our getCohort() function created in step 2. It then rewrites the response to show the correct page to the visitors given cohort. Last, if there isn't a cookie and we had to get it from our getCohort() function, we send the experiment cookie with the response so subsequent requests from the browser show the same page.

Now that our middleware is set up, we'll set up the homepage to render our experiment.

Step 4: The Homepage

Now we'll need to set up the homepage where the test will run. This page is dynamic so we'll need to rename the index.js file in your pages directory to [exp001].js. This takes advantage of Nextjs' dynamic routing. To render the correct page, we need to use getStaticPaths to define the lists of paths to be rendered. First, we'll need to import the cohorts that we created in Step 2.

import { COHORTS } from '../experiment/exp001';

Next, we need to add a getStaticPaths() function to loop through each cohort to define a path for each cohort page to be rendered to HTML at build time. We pass along the exp001 object which contains the cohort as params for the path.

export async function getStaticPaths() {
  return {
    paths: COHORTS.map((exp001) => ({ params: { exp001 } })),
    fallback: false,
  };
}

Now that we have our paths set, let's see them in action. We'll import useRouter to see which cohort we are randomly assigned:

import { useRouter } from 'next/router';

Then, declare the router and create a cohort constant from the router path:

const router = useRouter();
const cohort = router.query.exp001;

In the body, we'll render the current cohort in a <pre> tag

...
<div className="p-4">
  <pre>{cohort}</pre>
</div>
...

Your [exp001].js page should now look like this:

import { useRouter } from 'next/router';
import Head from 'next/head';
import { COHORTS } from '../experiment/exp001';

export default function Cohort() {
  const router = useRouter();
  const cohort = router.query.exp001;

  return (
    <div className="flex flex-col items-center justify-center min-h-screen py-2">
      <Head>
        <title>Simple Vercel Edge Functions A/B Test</title>
        <link rel="icon" href="/favicon.ico" />
        <meta
          name="description"
          content="An example a/b test app built with NextJs using Vercel edge functions"
        />
      </Head>

      <main className="flex flex-col items-center justify-center w-full flex-1 px-20 text-center">
        <h1 className="text-6xl font-bold">
          Vercel Edge Functions{' '}
          <span className="bg-gradient-to-r from-purple-700 to-blue-600 text-transparent bg-clip-text font-bold">
            A/B Test Example
          </span>{' '}
          With Amplitude
        </h1>
        <div className="p-4">
          <pre>{cohort}</pre>
        </div>
      </main>
    </div>
  );
}

export async function getStaticPaths() {
  return {
    paths: COHORTS.map((exp001) => ({ params: { exp001 } })),
    fallback: false,
  };
}

Start your local server with npm run dev and you should see the current cohort + experiment cookie in the dev tools.

When you refresh you'll notice you still see the same cohort - that's because the subsequent requests are receiving the experiment cookie already set in the browser. This is so your visitors are bucketed into the same cohort on any page refreshes or subsequent visits. To reset the cohort, we create a function and button to remove the experiment button to the middleware runs the getCohort() function on any new request when the reset cohort button is clicked:

npm i js-cookie

import Cookies from 'js-cookie'
...
  const removeCohort = () => {
    // removes experiment cookie
    Cookies.remove('exp001-cohort');
    // reloads the page to run middlware
    // and request a new cohort
    router.reload();
  };
  ...
  <button type="button" onClick={removeCohort}>
    Reset Cohort
    </button>
...

Now when you click the reset cohort button, you'll see the cohort switch depending on the random number returned from our getCohort() function.

Full [exp001].js code:

import { useRouter } from 'next/router';
import Head from 'next/head';
import Cookies from 'js-cookie';
import { COHORTS } from '../experiment/exp001';

export default function Cohort() {
  const router = useRouter();
  const cohort = router.query.exp001;

  const removeCohort = () => {
    // removes experiment cookie
    Cookies.remove('exp001-cohort');
    // reloads the page to run middlware
    // and request a new cohort
    router.reload();
  };

  return (
    <div className="flex flex-col items-center justify-center min-h-screen py-2">
      <Head>
        <title>Simple Vercel Edge Functions A/B Test</title>
        <link rel="icon" href="/favicon.ico" />
        <meta
          name="description"
          content="An example a/b test app built with NextJs using Vercel edge functions"
        />
      </Head>

      <main className="flex flex-col items-center justify-center w-full flex-1 px-20 text-center">
        <h1 className="text-6xl font-bold">
          Vercel Edge Functions{' '}
          <span className="bg-gradient-to-r from-purple-700 to-blue-600 text-transparent bg-clip-text font-bold">
            A/B Test Example
          </span>{' '}
          With Amplitude
        </h1>
        <div className="p-4">
          <pre>{cohort}</pre>
        </div>

        <button type="button" onClick={removeCohort}>
          Reset Cohort
        </button>
      </main>
    </div>
  );
}

export async function getStaticPaths() {
  return {
    paths: COHORTS.map((exp001) => ({ params: { exp001 } })),
    fallback: false,
  };
}

Now we have a functioning site that assigns a cohort to each user. In part 2, we'll create the test button, render the correct button, and cover how to track our experiment analytics using Amplitude!

Part 1: Github Repo
Part 1: Site Example

Want to collect feedback on your A/B test? Start collecting feedback in 5 minutes with SerVoice!