DEV Community: Jeremy Dorn

Solving cache invalidation in 15 lines of code

Jeremy Dorn — Sun, 27 Feb 2022 08:14:45 +0000

Caching lets you manage the trade-off between freshness and speed when dealing with slow asynchronous calls.

If you only care about freshness, it's easy. Just set a really short cache time (or forego caching entirely). The data will always be up-to-date, but you'll constantly be waiting on the slow calls to finish.

If you only care about speed, it's also easy. Just set a super long cache time. You'll rarely have to wait for the slow calls, but the data may be really out-of-date.

The problem is, developers often want both - fresh data as fast as possible. Cache invalidation is one way to achieve this, but it's incredibly hard to do well. In fact, there's a common saying:

The two hardest problems in Computer Science are naming, cache invalidation, and off-by-one errors.

Stale-while-revalidate (or SWR) is another attempt to solve this problem that is much simpler.

When a stale item is requested from the cache, you immediately return it (maximum speed). You then kick off a background task to update the value (maximum freshness).

In fact, it's so simple, we can implement it in only 15 lines of javascript code:

const cache = new Map();
async function swr(key, refresh, staleAfter = 5000) {
  let data = cache.get(key) || {ts: 0, val: null, promise: null}
  cache.set(key, data);
  // Item is stale, start refreshing in the background
  if (!data.promise && Date.now() - data.ts > staleAfter) {
    data.promise = refresh()
      .then((val) => { data.ts = Date.now(); data.val = val; })
      .catch((e) => console.error(e))
      .finally(() => (data.promise = null));
  }
  // No data yet, wait for the refresh to finish
  if (data.promise && !data.ts) await data.promise;
  return data.val;
}

And this is how you would use it:

const data = await swr("cache-key", expensiveFuncion, 5000)

The first time you run that code, it will wait for the expensive function to finish. Subsequent calls will always return immediately. If the returned value is stale (more than 5 seconds old), it will refresh it in the background.

You can get a lot fancier with the implementation, but the basics of SWR really are that simple!

If you are building a React App and want to use this method to fetch data for your front-end, I highly recommend the useSWR React hook by Vercel.

A/B Testing with Create React App

Jeremy Dorn — Thu, 10 Feb 2022 20:48:05 +0000

Create React App (CRA) is an extremely popular framework for building front-end applications.

Legacy A/B testing tools like Optimizely, VWO, and Google Optimize were built during the jQuery days and fall flat in modern React applications. Those tools work by applying changes on top of the DOM (e.g. dynamically changing the src of an <img> tag). That's exactly how React works as well, which is where the problems start. These tools end up in an arms race where React will re-render a component and then the A/B testing script has to quickly switch it back before the user notices. This requires massive complexity and results in huge (100Kb+) javascript files that are slow and fragile.

GrowthBook is a new open source A/B testing platform built from the ground up for modern tech stacks. By leveraging the power of React (instead of fighting against it), GrowthBook is able to provide a really powerful and flexible A/B testing library in under 3Kb.

1. Set up Create React App

One of the best things about Create React App is how simple it is to get a new project started. We'll do the standard process with one extra command to install the GrowthBook SDK and the nanoid library:

npx create-react-app my-app
cd my-app
npm install --save @growthbook/growthbook-react nanoid
npm start

Then open http://localhost:3000/ and you're good to go.

2. Set up GrowthBook

In this tutorial, I'm going to use the cloud-hosted version of GrowthBook, which is free for small teams, but you can also use the open source version and host it yourself if that's more your style.

GrowthBook uses the concept of Feature Flagging to run A/B tests. Basically, you wrap the code you want to test in a conditional check if (feature.on) {...} and then you run an A/B test within GrowthBook to turn the feature on for 50% of users and off for the other 50%.

To start, go to https://app.growthbook.io and register a new account. Then there are a couple steps required before you can run an experiment.

3. Install and configure the SDK

Next, click on Step 1: Install our SDK and you should see API keys for dev/production as well as sample code.

We already ran the npm i command at the start, so you can skip that part. We can use the React code example as a starting point. I'll walk through the different parts below:

First, in src/index.js, we import the GrowthBook SDK and nanoid library:

// ... after existing imports
import { 
  GrowthBook, 
  GrowthBookProvider 
} from "@growthbook/growthbook-react";
import { nanoid } from "nanoid";

Then we need to generate an anonymous visitor id, which is used to assign an A/B test variation to a user. We'll persist this in localStorage so if the user refreshes our app they will get assigned the same variation as before.

let visitor_id = localStorage.get("visitor_id");
if(!visitor_id) {
  visitor_id = nanoid()
  localStorage.set("visitor_id", visitor_id);
}

Then, we create a GrowthBook instance with our visitor id and a tracking callback when a user is put into an experiment.

const growthbook = new GrowthBook({
  attributes: {
    id: visitor_id
  },
  trackingCallback: (experiment, result) => {
    console.log({
      experimentId: experiment.key, 
      variationId: result.variationId
    })
  }
});

After that, we fetch the list of features from the GrowthBook API and pass them into the SDK:

const FEATURES_ENDPOINT = 
  'https://cdn.growthbook.io/api/features/...';

fetch(FEATURES_ENDPOINT)
  .then((res) => res.json())
  .then((json) => {
    growthbook.setFeatures(json.features);
  });

Make sure to swap out the FEATURES_ENDPOINT constant above with your own dev API key you see in the GrowthBook application.

Lastly, we need to wrap the app in a GrowthBookProvider component which will let us run A/B tests from anywhere in the app.

ReactDOM.render(
  <React.StrictMode>
    <GrowthBookProvider growthbook={growthbook}>
      <App />
    </GrowthBookProvider>
  </React.StrictMode>,
  document.getElementById('root')
);

4. Create and use a feature

Now that the SDK is installed and fully integrated in our application, we can finally create the show-logo feature.

Back in GrowthBook, Click on Step 2: Add your first feature and fill in the following info:

Feature key - show-logo
Dev - toggle on
Prod - toggle off
Value Type - boolean (on/off)
Behavior - A/B Experiment
Tracking Key - show-logo
Sample Users based on attribute - id
Variations and Weights - leave default (OFF/ON, 50/50 split)
Fallback Value - OFF

There's a lot of fields there, but hopefully it's pretty straight forward what's happening. We setup a new boolean feature called show-logo, that's only enabled in dev and running an A/B test where 50% get ON and 50% get OFF

Now we can switch back to our React app and reference this feature in our code.

In src/App.js, we currently have the following code:

<img src={logo} className="App-logo" alt="logo" />

Let's add an import at the top of the file:

import { IfFeatureEnabled } from '@growthbook/growthbook-react';

And wrap the img element in an IfFeatureEnabled component:

<IfFeatureEnabled feature="show-logo">
  <img src={logo} className="App-logo" alt="logo" />
</IfFeatureEnabled>

Now, if you refresh your app, the A/B test should be running! If you're part of the lucky 50% that are in the B variation (no logo), it should be pretty obvious. If you happen to be in the A variations, you can verify you're in the test by looking in DevTools for our trackingCallback console.log.

You can test out different variations by deleting the visitor_id from localStorage and refreshing your app. Repeat a few times and you should see each version of the page about half of the time. If you want an easier and faster way to QA the variations, you can download the GrowthBook Chrome DevTools Extension.

5. Analyze Results

Just running an A/B test by itself is fun, but not that useful. You also need to track and analyze the results.

In the trackingCallback in src/index.js, instead of doing a console.log, we could use Mixpanel or Segment or another event tracking system.

Then, throughout our app, we could similarly track events when users do something we care about, like sign up or buy something.

Once you do that, GrowthBook can connect to your event tracking system, query the raw data, run it through a stats engine, and show you the results. This process is a little more involved to set up, but I will walk through it in a followup post.

Next Steps

There's so much more you can do with GrowthBook beyond a simple on/off A/B test!

Add complex targeting and override rules for your features
Set up webhooks to keep a cached copy of features in your database
Use the useFeature React hook for more control in your code
Install the Chrome DevTools Extension to test different variations and scenarios
Read about the powerful statistics engine that is used to analyze experiment results.

Twyman's Law

Jeremy Dorn — Fri, 28 Jan 2022 03:21:07 +0000

Has something like this ever happened to you?

You see something crazy on a dashboard - your website traffic doubled today out of nowhere!

You rule out the usual suspects - no front page of reddit, Barack Obama didn't tweet about you. Something else must be going on.

You changed a button color this morning, could that be it?

Maybe people really like purple and are sharing links to your site with all of their friends! Or maybe Google decided to rank your site higher in search results because it looks so much nicer now!

Either way, you're about to be rich! Time to start looking on Zillow for the next house you're going to buy.

Wait a minute, you decide to double check that button color change you made earlier.

Oh crap! You notice a typo that's making every page view event fire twice!

Your traffic didn't actually double. The tracking was just broken!

Next time, $10M mansion, next time.

If this sounds at all familiar, you aren't alone.

Twyman's Law states that the more unusual or interesting a piece of data, the more likely that it's false.

It's in the same vein as "extraordinary claims require extraordinary evidence."

So next time you discover something amazing in your data, your first thought should be "why is this data wrong?" and not "which Hawaiian island am I going to retire to?"

Feature Flags with Next.js and GrowthBook

Jeremy Dorn — Fri, 31 Dec 2021 12:05:08 +0000

Deploying code to production is scary and stressful. There's nothing worse than seeing an unexpected spike or dip in your metrics as soon as your code goes live. Your heart is racing as you quickly revert the commit. The site is broken and there's nothing you can do but wait 5+ agonizing minutes for your CI/CD pipeline to finish. And that's the best case scenario when a simple code revert is possible. Need to deal with rolling back database or infrastructure changes? Good luck with that.

Photo by ETA+ on Unsplash

Feature flags to the rescue!

Luckily, there's a better way to deploy code thanks to Feature Flags. With feature flags, your deploy is easy - the new code goes live, but everyone just continues getting the old experience initially. Feature flags decouple deployment and release. It's still possible to break things during deploy, but a lot less likely when no one is seeing the code right away.

Using remote configuration, you are able to instantly update which version of the code users see without a deploy. Maybe you start by only enabling it for you and your teammates while you click around and see if it's working. Then you can gradually roll it out to 100% of your users while you watch your metrics. If you do notice a bug, you can instantly just switch everyone back to the old code while you take your time debugging and fixing the problem.

By using feature flags, you just decreased your downtime from 5+ minutes to mere seconds and limited the impact to a small percent of your traffic. Plus, you decreased the stress levels of you and your teammates 100 fold, which is always a good thing.

Tutorial

In this tutorial, I'm going to walk through how to add feature flags as part of your development process. I'll be using a standard Next.js app and GrowthBook as the feature flagging platform.

GrowthBook is better known as an open source A/B testing platform, but it is also very capable at feature flagging. Other popular open source options are Unleash and FlagSmith. For paid platforms, there is LaunchDarkly and Split.io, although these can get prohibitively expensive for many teams.

Step 1: Next.js App

Let's start by getting a basic Next.js app running:



yarn create next-app

Then cd into the newly create directory and run:



cd my-app
yarn dev -p 4000

Note: Both GrowthBook and Next.js run on port 3000 by default, so we're making our Next.js app use 4000 instead to avoid conflicts.

Visit http://localhost:4000 and you should see the application running!

Step 2: GrowthBook Account

For this tutorial, we'll run GrowthBook locally, but you can also use a free cloud-hosted account at https://app.growthbook.io instead if you want.



git clone https://github.com/growthbook/growthbook.git
cd growthbook
docker-compose up -d

After that, visit http://localhost:3000 and create your first user account.

Step 3: Integrate the GrowthBook React SDK into our Next.js app

GrowthBook will generate some integration code for you, including a unique API endpoint to load your features from.

We first need to install the GrowthBook React SDK in our Next.js app:



yarn add @growthbook/growthbook-react

Then we can modify the generated React code to work with the Next.js framework. Modify the file pages/_app.js with the following contents:



import '../styles/globals.css'
import { 
  GrowthBook, 
  GrowthBookProvider 
} from "@growthbook/growthbook-react";
import { useEffect } from "react";

const FEATURES_ENDPOINT = 
  "http://localhost:3100/api/features/key_abc123";

// Create a GrowthBook instance
const growthbook = new GrowthBook({
  trackingCallback: (experiment, result) => {
    console.log("Viewed Experiment", experiment, result);
  }
})

export default function MyApp({ 
  Component, 
  pageProps, 
  router 
}) {
  // Refresh features and targeting attributes on navigation
  useEffect(() => {
    fetch(FEATURES_ENDPOINT)
      .then((res) => res.json())
      .then((json) => {
        growthbook.setFeatures(json.features);
      });

    growthbook.setAttributes({
      "id": "123",
      "loggedIn": true,
      "deviceId": "abcdef123456",
      "employee": true,
      "company": "acme",
      "country": "US",
      "browser": navigator.userAgent,
      "url": router.pathname
    })
  }, [router.pathname])

  return (
    <GrowthBookProvider growthbook={growthbook}>
      <Component {...pageProps} />
    </GrowthBookProvider>
  )
}

Replace the FEATURES_ENDPOINT constant with the one you see in your instructions modal in GrowthBook.

In a real application, you would pull some of the targeting attributes from your authentication system or an API, but let's just leave them hard-coded for now.

Step 4: Create a Feature in GrowthBook

Back in the GrowthBook application, we can create a new feature. For this tutorial, we'll make a simple on/off feature flag that determines whether or not we show a welcome banner on our site.

The key we chose (welcome-message) is what we will reference when using the GrowthBook SDK.

We can now edit pages/index.js and conditionally render a welcome message based on the state of the feature:

Add an import statement:



import { IfFeatureEnabled } from "@growthbook/growthbook-react";

And then put your welcome message somewhere on the page:



  <IfFeatureEnabled feature="welcome-message">
    <p>I hope you enjoy this site and have a great day!</p>
  </IfFeatureEnabled>

If you refresh your Next.js app, you'll notice the welcome message is not rendered. This is because when creating the feature, we set the default value to off. At this point, we could safely deploy our change to production and not worry about breaking anything.

Step 5: Turn on the feature for your team

Now we can add rules to the feature to turn it on for specific groups of users.

In the hard-coded targeting attributes we set in pages/_app.js, we designated ourselves as an internal employee. Let's use this attribute to turn on the feature for the whole internal team:

Refresh your Next.js app and you should now see the welcome message appearing! (Note: it might take up to 30 seconds for the API cache to refresh).

If you change employee to false in pages/_app.js, you should see the welcome message disappear.

The best part about targeting attributes in GrowthBook is that they are evaluated entirely locally. Sensitive user data is never sent over the network and there is no performance penalty. Some other libraries require an HTTP request to evaluate a feature for a user and this is often a deal breaker.

Step 6: Gradually roll out to your users

After you test the new feature within your team, you probably want to start rolling it out to real users.

We can do that with another rule in GrowthBook:

In the targeting attributes in pages/_app.js, make sure employee is set to false. That will ensure you skip the first rule we made and fall into the second rollout rule.

The GrowthBook SDK uses deterministic hashing to figure out whether or not someone is included in a rollout. Basically, it hashes together the selected targeting attribute (id) and the feature key (welcome-message) and coverts it to a float between 0 and 1. If that float is less than or equal to the rollout percent, the user is included. This ensures a consistent UX and prevents one user from constantly switching between ON and OFF as they navigate your app.

Try changing the user id in the targeting attributes in pages/_app.js to a few different random strings and see what happens. You should notice about half of the time the welcome message shows up and half of the time it doesn't. The rollout is working!

Conclusion and Next Steps

Now you've seen how to set up and integrate GrowthBook into your Next.js application, use feature flags in your code, remotely enable the feature in GrowthBook based on targeting conditions, and gradually roll out the feature to your users.

Once you do the initial integration work, it only takes a few seconds to wrap your code in feature flags. Once you realize how easy and stress-free deploys can be, there's no going back.

Next Steps:

Look at the GrowthBook React SDK docs for more ways to use feature flags in your code besides the <IfFeatureEnabled> component
Read more about using features in GrowthBook, including how to use complex feature values (not just ON/OFF), run A/B tests, and analyze the impact of your features using data.

Happy feature flagging!

Beware of Promise.all

Jeremy Dorn — Thu, 28 Oct 2021 18:26:46 +0000

In Javascript, Promise.all lets you execute a bunch of Promises in parallel and get an array of results back.

const responses = await Promise.all([
  fetch("/api/1"),
  fetch("/api/2")
])

Pretty straight forward. However, if you were to do the above with 100 fetch calls instead, you might accidentally take down your server in a self-inflicted Denial of Service attack. Even if you protect against this in the API with rate-limiting, you're still going to see a lot of errors for failed requests as you scale up.

APIs are the exception. Most types of external calls have no concept of rate-limiting at all - filesystem operations, system calls, etc.

For example, in NodeJS you can spawn new shells to call out to other programs on the computer. I use this in my open source A/B testing platform GrowthBook to call a Python script. Something like this:

const results = await Promise.all(
  metrics.map(m => callPython(m))
)

The above will happily spawn hundreds of Python shells if given a large array and start executing them all in parallel. My dev machine is pretty powerful, so I didn't notice during testing that all 8 CPU cores would go to 100% for a couple seconds. When I deployed the code to a Docker container on AWS though, I definitely noticed when it started crashing and restarting all the time.

The solution is to add rate-limiting or concurrency limits to your Promise.all calls. There are a few ways to do this.

For API calls where you want to limit the number of calls per second, you can use the simple p-throttle library:

import pThrottle from 'p-throttle'

// Limit to 2 calls per second
const throttle = pThrottle({
  limit: 2,
  interval: 1000
})

const responses = await Promise.all([
  throttle(() => fetch("/api/1")),
  throttle(() => fetch("/api/2")),
  ...
])

For system calls where you want to limit the number of parallel executions, no matter how long they take, there is the simple p-limit library:

import pLimit from 'p-limit'

// Only 5 promises will run at a time
const limit = pLimit(5)

const results = await Promise.all(
  metrics.map(
    m => limit(() => callPython(m))
  )
)

For more advanced use cases, you might want to look into using a full-featured job queue instead like bree, bull, or agenda.

As developers we spend a lot of time worrying about external attacks and not enough time on protecting our apps from naive internal code. I hope this helps others avoid the same CPU crashing bugs in production that I had to work through. Good luck out there!

A/B Testing with the new Next.js 12 Middleware

Jeremy Dorn — Tue, 26 Oct 2021 20:34:54 +0000

Vercel recently released Next.js 12 which adds a number of exciting performance improvements as well as a new beta feature - Middleware. Middleware has many uses, but I'm going to focus in this post on A/B Testing.

You've always been able to run A/B tests on Next.js applications, but until this latest release there have been some major strings attached. For example, on static pages, there would often be a "flash" where users would see the original page for a split second before your variation popped in and replaced it. And on server-rendered pages, you would need to completely disable caching since two users on the same URL could get two different HTML responses.

Next.js middleware fixes these issues in an elegant way. You can create two different versions of a page and using a single URL, route traffic between them with a middleware function. The middleware is run at the edge, so it's globally distributed and super fast for your users.

Setting up the Next.js App

We'll start with a standard Typescript Next.js app:

npx create-next-app@latest --ts
cd my-app
npm run dev

Now you should be able to visit http://localhost:3000 and see a homepage.

Let's create a new version of the homepage at pages/new_homepage.tsx:

export default function NewHomePage() {
  return <h1>Welcome to my new site!</h1>
}

Now you should have two working URLs:

The original homepage - http://localhost:3000
The new (way better) homepage - http://localhost:3000/new_homepage

Our goal is instead of having these on two separate URLs, we want 50% of the visitors to the root URL (/) to see the original page and the other 50% to see the new one.

That sounds like an A/B test! To help do the traffic splitting, we're going to use GrowthBook, an open source feature flagging platform with really robust A/B testing support.

Setting up GrowthBook

You can self-host GrowthBook (https://github.com/growthbook/growthbook) or create a free Cloud account (https://app.growthbook.io/). Either way, once you login, there are a couple steps required before you can run an experiment.

First, click on Step 1: Install our SDK and run the npm install command:

npm i --save @growthbook/growthbook

Note: Next.js middleware runs outside of a React context, so we're using the vanilla Javascript SDK above instead of the React one.

Creating the Next.js Middleware

Now, we'll integrate the sample code in GrowthBook into our Next.js App. Create a file pages/_middleware.ts with the following contents (make sure to swap out the placeholder with the unique API endpoint you see in GrowthBook):

import { NextRequest, NextResponse } from 'next/server'
import { GrowthBook } from '@growthbook/growthbook'

const FEATURES_ENDPOINT = 'YOUR_GROWTHBOOK_ENDPOINT_HERE'

// Fetch features from GrowthBook API and cache in memory
let features = null;
let lastFetch = 0;
async function getFeatures() {
  if (Date.now() - lastFetch > 5000) {
    lastFetch = Date.now();
    const latest = fetch(FEATURES_ENDPOINT)
      .then(res => res.json())
      .then(json => features = json.features || features)
      .catch((e) => console.error("Error fetching features", e))
    // If this is the first time, wait for the initial fetch
    if(!features) await latest;
  }
  return features || {};
}

const COOKIE = 'visitor_id'

export async function middleware(req: NextRequest) {
  // We only want to run the A/B test on the homepage
  const pathname = req.nextUrl.pathname;
  if (pathname !== "/") {
    return NextResponse.next()
  }

  // Get existing visitor cookie or create a new one
  let visitor_id = req.cookies[COOKIE] || crypto.randomUUID()

  // Create a GrowthBook client instance
  const growthbook = new GrowthBook({
    attributes: { id: visitor_id },
    features: await getFeatures(),
    trackingCallback: (exp, res) => {
      console.log("In Experiment", exp.key, res.variationId);
    }
  });

  // Pick which page to render depending on a feature flag
  let res = NextResponse.next();
  if (growthbook.feature("new-homepage").on) {
    const url = req.nextUrl.clone();
    url.pathname = "/new_homepage";
    res = NextResponse.rewrite(url);
  }

  // Store the visitor cookie if not already there
  if (!req.cookies[COOKIE]) {
    res.cookie(COOKIE, visitor_id)
  }

  return res
}

There's a lot going on here, but it's not too hard to follow:

Function to fetch feature definitions from the GrowthBook API, cache them, and keep it up-to-date
Skip the middleware if the user is requesting any page other than /
Look for an existing visitor id stored in a cookie and generate one if it doesn't exist yet.
Create a GrowthBook client instance
Determine which page to render based on a GrowthBook feature flag
Set the visitor id cookie on the response if needed
Return the response

Creating the Feature Flag

At this point, if you visit http://localhost:3000 you'll always see the original homepage still.

This is because the code is looking for a feature flag named new-homepage, which doesn't exist yet. Flags that don't exist yet are always treated as if they are off so the middleware just returns the original page. Let's fix this.

Back in GrowthBook, close the SDK instructions modal and click on Step 2: Add your first feature. Enter the feature key new-homepage. Keep the feature type set to on/off, choose "A/B Experiment" as the behavior, and leave everything else set to the default (split users by id, even 50/50 split, "new-homepage" as the tracking key).

Click save, wait a few seconds, and refresh your NextJS app.

Depending on the random visitor_id cookie that the middleware generated, you may see either version of the homepage. You can delete that cookie and refresh a few times. You'll notice about half the time you get the new page and the other half you don't.

Also, if you look in the terminal where you're running the Next.js npm run dev command, you should see the log messages from trackingCallback.

Analyzing Results

Just running an A/B test by itself is fun, but not that useful. You also need to track and analyze the results.

In the trackingCallback in pages/_middleware.ts, instead of doing a console.log, we could use Mixpanel or Segment or another event tracking system.

Then, in the app, we could similarly track events when the users do something we care about, like sign up or buy something.

The Future

Hopefully in future releases, Next.js expands on their middleware feature to make A/B testing even more powerful. Imagine, for example, that middleware could inject props into your pages, similar to getServerSideProps. Then you wouldn't need to create new temporary pages every time you wanted to run an A/B test!