DEV Community: David Groechel

NLP address parser: From concept to production MVP in a weekend

David Groechel — Tue, 26 Apr 2022 23:13:42 +0000

Last week I came across two problems that intersected.

Users not coming back to my app because it's not top of mind when they sign up. Then they get busy planning, making them less likely to come back.
Collecting addresses for a wedding

I decided to come up with a solution - an address parser using NLP. This would help solve the log-in once issue by becoming the system of record for weddings. It also solves the issue of collecting addresses. You need them to send save the dates, invites, and thank you cards. They are a pain to gather and keep up to date, especially if you have 100+ guests. We ended up texting our friends and family to gather the addresses. While we were able to gather them, the manual entry was time-consuming. I decided to investigate adding an address parser to Text My Wedding using Spacy and AWS Lambdas. I'll go over the how and why I chose this route.

Wait, before we start, what is Spacy and named entity recognition?

Spacy is an open sourced library for Natural Language Processing.

"spaCy is designed specifically for production use and helps you build applications that process and “understand” large volumes of text. It can be used to build information extraction or natural language understanding systems, or to pre-process text for deep learning." - Spacy.io

Named entity recognition is using a model to recognize and predict named entities - anything assigned a name like people, products, places, etc. NER is helpful in pulling out these entities from a large volume of text you wan't to analyze.

Heres an example of Spacy pulling out ORG, GPE, and MONEY entities.

Got it, so how did you build this?

I decided to run this on AWS lambda. You may be asking "why in the world would you run a serverless NLP application?" That's a very fair question and I decided this for a few reasons:

Go with what you know. It's as simple as that - I know my way around Lambdas pretty well. No reason to reinvent the wheel and try to learn other tools even if they are better suited for the long term.
It allowed me to get up and running in a weekend. I was able to spend my time learning how to train a custom model with Spacy. I didn't need to worry about spinning up servers or maintaining them. The rest of the time was learning the nuances of Python as a javascript developer. That's a story for another day...
I'm not 100% sure how users will adopt the feature. I'll let the photo below speak for itself...

What does it look like?

A Lambda is used to process the Twilio webhook events for incoming messages. This function updates the DB with the incoming message info. It then sends an SNS message to the Python lambda with the Spacy model. If the model recognizes an address in a message, it will send an SNS message to another Lambda. This Lambda then validates the parsed address using AWS Location Service and updates the DB with address info.

So is this scalable?

Absolutely not! But that wasn't the point of adding this feature. I wanted to quickly and cheaply build something to validate the address parser idea. It will also allow me to collect more training data and further improve the parsing. If users adopt the feature and we gather more training data, it will be time to move to a scalable solution.

Great, this works for now but what does a 'complex model' look like?

If users adopt the parser, the goal is to train a more advanced model. Right now, it will only process the text if there is one address. There are some cases to solve if a text contains two addresses. For example, if someone responds back with:

"Hey there! I live at 123 Main st now but am moving in 6 weeks to 144 Beacon st".

The model then needs to:

Classify there are two addresses
Recognize both addresses
Determine the order. Is the new or old address in position 1 or 2?

This is a much more complex use case. More data and tons more exploration is needed before we get there. Once there, better solutions would be to use SageMaker Serverless Inference or Fargate. Will I get there? That's to be determined by how users adopt the feature.

In the end, don't be afraid to just ship and figure out how to scale later. Thats half the fun of running a small startup. I was able to build something useful in a weekend and learn new skills.

Have questions or feedback? Feel free to leave a comment.

Part 1: Create an A/B test with Nextjs, Vercel edge functions, and measure analytics with amplitude

David Groechel — Fri, 05 Nov 2021 17:10:02 +0000

You're getting tons of traffic to your website but conversions aren't great. You decide to run an A/B test to see if you can increase your pageview -> sign-up metrics. In this tutorial, we'll go over how to set up a simple A/B test with Nextjs, Vercel edge functions, and measure analytics with amplitude.

Part 1: Github Repo
Part 1: Site Example

Note: We will not get into good experimentation practices in this article. This tutorial is only to show how to effectively set up an A/B test using edge functions and measure with amplitude

Step 1: Create a new Nextjs app

npx create-next-app -e with-tailwindcss feedback-widget

Open up the new app in your code editor and we'll start building out our test!

Step 2: Setting up your experiment

Next, we'll need to set up the experiment. We decide to test button color (purple vs blue) to see if we can increase conversions. This is our first experiment so well name it exp001 and our experiment cohorts exp001-control (purple button) and exp001-variant (blue button).

Note: We choose button color to keep it simple for this test. Changing a color is generally not an acceptable A/B test to run.

Create an experiment folder in your project. Within the experiment folder, we'll need two files ab-testing.js and exp001.js.

Stetting up the cohorts

We have already decided on our two cohorts and their names for the experiment. These need to be set up as constants to use throughout the project. In your exp001.js file, we will name the cohorts and cookie:

// experiment cohort names
export const COHORTS = ['exp001-control', 'exp001-variant'];
// experiment cookie name
export const COOKIE_NAME = 'exp001-cohort';

Splitting traffic

Now that we have our cohorts, in our ab-testing file, we will set up our traffic splittitng. At the top of the file, create a function to generate a random number:

function cryptoRandom() {
  return (
    crypto.getRandomValues(new Uint32Array(1))[0] / (0xffffffff + 1)
  );
}

In our case we use crypto.getRandomValues() - you can always use Math.random() (we won't debate the differences between the two in this tutorial - follow good practice and use what you know best!). This function will give us a random number between 0 & 1. Next, create a function that names the cohort based on the random number above:

export function getCohort(cohorts) {
  // Get a random number between 0 and 1
  let n = cryptoRandom() * 100;
  // Get the percentage of each cohort
  const percentage = 100 / cohorts.length;
  // Loop through the cohors and see if the random number falls
  // within the range of the cohort
  return (
    cohorts.find(() => {
      n -= percentage;
      return n <= 0;
      // if error fallback to control
    }) ?? cohorts[0]
  );
}

The getCohorts() function above breaks the cohorts into an even split depending on the number of cohorts.

Now that we have our cohorts and traffic splitting function. We'll set up our homepage for the test.

Step 3: Middleware

What is middleware at the edge?

Vercel edge functions allow you to deploy middleware to the edge - close to your visitor's origin. Middleware is the actual code the runs before a request is processed. You can execute many different functions using middleware such as running an A/B test as we are here, blocking bots, and redirects just to name a few. The middleware function runs before any requests to your pages are completed.

Setting up our traffic splitting middleware

To run middleware we need to create a _middleware.js file in our pages directory. This middleware will run before any page request is completed.

import { getCohort } from '../experiment/ab-testing';
import { COHORTS, COOKIE_NAME } from '../experiment/exp001';

export function middleware(req) {
  // Get the cohort cookie
  const exp001 = req.cookies[COOKIE_NAME] || getCohort(COHORTS);
  const res = NextResponse.rewrite(`/${exp001}`);

  // For a real a/b test you'll want to set a cookie expiration
  // so visitors see the same experiment treatment each time
  // they visit your site

  // Add the cohort name to the cookie if its not there
  if (!req.cookies[COOKIE_NAME]) {
    res.cookie(COOKIE_NAME, exp001);
  }

  return res;
}

The middleware first attempts to get the cohort cookie if there is one and if not, runs our getCohort() function created in step 2. It then rewrites the response to show the correct page to the visitors given cohort. Last, if there isn't a cookie and we had to get it from our getCohort() function, we send the experiment cookie with the response so subsequent requests from the browser show the same page.

Now that our middleware is set up, we'll set up the homepage to render our experiment.

Step 4: The Homepage

Now we'll need to set up the homepage where the test will run. This page is dynamic so we'll need to rename the index.js file in your pages directory to [exp001].js. This takes advantage of Nextjs' dynamic routing. To render the correct page, we need to use getStaticPaths to define the lists of paths to be rendered. First, we'll need to import the cohorts that we created in Step 2.

import { COHORTS } from '../experiment/exp001';

Next, we need to add a getStaticPaths() function to loop through each cohort to define a path for each cohort page to be rendered to HTML at build time. We pass along the exp001 object which contains the cohort as params for the path.

export async function getStaticPaths() {
  return {
    paths: COHORTS.map((exp001) => ({ params: { exp001 } })),
    fallback: false,
  };
}

Now that we have our paths set, let's see them in action. We'll import useRouter to see which cohort we are randomly assigned:

import { useRouter } from 'next/router';

Then, declare the router and create a cohort constant from the router path:

const router = useRouter();
const cohort = router.query.exp001;

In the body, we'll render the current cohort in a <pre> tag

...
<div className="p-4">
  <pre>{cohort}</pre>
</div>
...

Your [exp001].js page should now look like this:

import { useRouter } from 'next/router';
import Head from 'next/head';
import { COHORTS } from '../experiment/exp001';

export default function Cohort() {
  const router = useRouter();
  const cohort = router.query.exp001;

  return (
    <div className="flex flex-col items-center justify-center min-h-screen py-2">
      <Head>
        <title>Simple Vercel Edge Functions A/B Test</title>
        <link rel="icon" href="/favicon.ico" />
        <meta
          name="description"
          content="An example a/b test app built with NextJs using Vercel edge functions"
        />
      </Head>

      <main className="flex flex-col items-center justify-center w-full flex-1 px-20 text-center">
        <h1 className="text-6xl font-bold">
          Vercel Edge Functions{' '}
          <span className="bg-gradient-to-r from-purple-700 to-blue-600 text-transparent bg-clip-text font-bold">
            A/B Test Example
          </span>{' '}
          With Amplitude
        </h1>
        <div className="p-4">
          <pre>{cohort}</pre>
        </div>
      </main>
    </div>
  );
}

export async function getStaticPaths() {
  return {
    paths: COHORTS.map((exp001) => ({ params: { exp001 } })),
    fallback: false,
  };
}

Start your local server with npm run dev and you should see the current cohort + experiment cookie in the dev tools.

When you refresh you'll notice you still see the same cohort - that's because the subsequent requests are receiving the experiment cookie already set in the browser. This is so your visitors are bucketed into the same cohort on any page refreshes or subsequent visits. To reset the cohort, we create a function and button to remove the experiment button to the middleware runs the getCohort() function on any new request when the reset cohort button is clicked:

npm i js-cookie

import Cookies from 'js-cookie'
...
  const removeCohort = () => {
    // removes experiment cookie
    Cookies.remove('exp001-cohort');
    // reloads the page to run middlware
    // and request a new cohort
    router.reload();
  };
  ...
  <button type="button" onClick={removeCohort}>
    Reset Cohort
    </button>
...

Now when you click the reset cohort button, you'll see the cohort switch depending on the random number returned from our getCohort() function.

Full [exp001].js code:

import { useRouter } from 'next/router';
import Head from 'next/head';
import Cookies from 'js-cookie';
import { COHORTS } from '../experiment/exp001';

export default function Cohort() {
  const router = useRouter();
  const cohort = router.query.exp001;

  const removeCohort = () => {
    // removes experiment cookie
    Cookies.remove('exp001-cohort');
    // reloads the page to run middlware
    // and request a new cohort
    router.reload();
  };

  return (
    <div className="flex flex-col items-center justify-center min-h-screen py-2">
      <Head>
        <title>Simple Vercel Edge Functions A/B Test</title>
        <link rel="icon" href="/favicon.ico" />
        <meta
          name="description"
          content="An example a/b test app built with NextJs using Vercel edge functions"
        />
      </Head>

      <main className="flex flex-col items-center justify-center w-full flex-1 px-20 text-center">
        <h1 className="text-6xl font-bold">
          Vercel Edge Functions{' '}
          <span className="bg-gradient-to-r from-purple-700 to-blue-600 text-transparent bg-clip-text font-bold">
            A/B Test Example
          </span>{' '}
          With Amplitude
        </h1>
        <div className="p-4">
          <pre>{cohort}</pre>
        </div>

        <button type="button" onClick={removeCohort}>
          Reset Cohort
        </button>
      </main>
    </div>
  );
}

export async function getStaticPaths() {
  return {
    paths: COHORTS.map((exp001) => ({ params: { exp001 } })),
    fallback: false,
  };
}

Now we have a functioning site that assigns a cohort to each user. In part 2, we'll create the test button, render the correct button, and cover how to track our experiment analytics using Amplitude!

Part 1: Github Repo
Part 1: Site Example

Want to collect feedback on your A/B test? Start collecting feedback in 5 minutes with SerVoice!

Create A Feedback Widget for Your React App

David Groechel — Tue, 19 Oct 2021 00:49:42 +0000

You've created a new web app, launched successfully and acquired new users. Now what? You wonder are there bugs I'm missing? Are users enjoying the app? What features do they want added?

This is a great time to add a feedback widget to your app. In Part 1, we'll show how to create a widget in react with NextJs, react-popper, and Tailwind

Setting up our project

Step 1: Create a new NextJs app

Note: If you already have a react app you can skip to step 2


 create-next-app -e with-tailwindcss feedback-widget

Open up the new app in your code editor, open up the pages/index.js file and delete out the contents. You should end up with:



import Head from "next/head";

export default function Home() {
  return (
    <div className="flex flex-col items-center justify-center min-h-screen py-2">
      <Head>
        <title>React Feedback Widget Example</title>
        <link rel="icon" href="/favicon.ico" />
      </Head>
      Hello!
    </div>
  );
}

Step 2: Create feedback button

For this tutorial, we'll create a floating action button to trigger the feedback popup. This button will "float" at the bottom right of your page and will always be visible for the visitor/user to leave feedback.

Note: You can create any type of button that fits your project.



import Head from "next/head";

export default function Home() {
  return (
    <div className="flex flex-col items-center justify-center min-h-screen py-2">
      <Head>
        <title>React Feedback Widget Example</title>
        <link rel="icon" href="/favicon.ico" />
      </Head>
      <button
        className="absolute bottom-4 right-4 p-2 rounded-l-lg rounded-t-lg bg-blue-700 hover:bg-blue-600 shadow-md hover:shadow-lg text-white"
        type="button"
      >
        Feedback?
      </button>
    </div>
  );
}

We should now have something that looks like this:

Next, we are ready to create the feedback widget.

Step 3: Install react-popper


 i react-popper @popperjs/core

After installing, we'll need to import both usePopper and useState at the top of our component.



import { useState } from 'react';
import { usePopper } from 'react-popper';

Next, we'll set the popper reference and element. Full Code:



import { useState } from "react";
import { usePopper } from "react-popper";
import Head from "next/head";

export default function Home() {
  const [referenceElement, setReferenceElement] = useState(null);
  const [popperElement, setPopperElement] = useState(null);
   const { styles, attributes } = usePopper(referenceElement, popperElement, {
    modifiers: [{ name: "offset", options: { offset: [10, 10] } }],
  });
  return (
    <div className="flex flex-col items-center justify-center min-h-screen py-2">
      <Head>
        <title>React Feedback Widget Example</title>
        <link rel="icon" href="/favicon.ico" />
      </Head>
      <button
        ref={setReferenceElement}
        className="absolute bottom-4 right-4 p-2 rounded-l-lg rounded-t-lg bg-blue-700 hover:bg-blue-600 shadow-md hover:shadow-lg text-white"
        type="button"
      >
        Feedback?
      </button>

      <div ref={setPopperElement}
style={styles.popper}
 {...attributes.popper}>
        Feedback Widget
    </div>
  );
}

We should now have both the button and popper element visible:

Now let's add some styles to the popper.

Step 4: Adding styles to the widget

Right now your widget doesn't have any styleing. We'll add some styling to make it look more like a widget.

Add the classname below to the popper element:


   className="bg-white p-2 rounded-md shadow-lg"

Feel free to add different styling to make it your own!

Step 5: Adding a feedback form to the widget

Now that we've created the widget, we'll need to create a form for the user to submit feedback. We want a text area where users can type their feedback, a label for the text area, and a submit button.



...
 <div
        className="bg-white p-2 rounded-md shadow-lg"
        ref={setPopperElement}
        style={styles.popper}
        {...attributes.popper}
      >
        <form>
          <label for="feedback">How can we improve?</label>
          <div className="flex flex-col space-y-2">
            <textarea
              id="feedback"
              className="p-2 h-24 w-64 border border-gray-400 rounded-md"
              type="textarea"
            ></textarea>
            <button
              className="p-2 ml-auto rounded-lg bg-blue-700 hover:bg-blue-600 shadow-md hover:shadow-lg text-white"
              type="submit"
            >
              Send Feedback
            </button>
          </div>
        </form>
      </div>
...

Now our widget should look like this:

Step 6: Showing and hiding the widget

So we have our widget but now need to hide it on button click. We'll use useState for this:



 const [showWidget, setShowWidget] = useState(false)

The feedback button will need a click handler to show and hide the widget:



... 
 <button
        onClick={() => setShowWidget(!showWidget)}
        ref={setReferenceElement}
        className="absolute bottom-4 right-4 p-2 rounded-l-lg rounded-t-lg bg-blue-700 hover:bg-blue-600 shadow-md hover:shadow-lg text-white"
        type="button"
      >
        Feedback?
      </button>
...

Now we'll need to conditionally show the widget when showWidget = true



      {showWidget && (
        <div
          className="bg-white p-2 rounded-md shadow-lg"
          ref={setPopperElement}
          style={styles.popper}
          {...attributes.popper}
        >
          <form>
            <label for="feedback">How can we improve?</label>
            <div className="flex flex-col space-y-2">
              <textarea
                id="feedback"
                className="p-2 h-24 w-64 border border-gray-400 rounded-md"
                type="textarea"
              ></textarea>
              <button
                className="p-2 ml-auto rounded-lg bg-blue-700 hover:bg-blue-600 shadow-md hover:shadow-lg text-white"
                type="submit"
              >
                Send Feedback
              </button>
            </div>
          </form>
        </div>
      )}

Now we have a "functioning" feedback widget built with react, react-popper, and tailwind!

Of course you'll need to create a backend to collect feedback and save it to your database. Don't have time for that? Use SerVoice and start collecting feedback within minutes.

Full Code:



import { useState } from "react";
import { usePopper } from "react-popper";
import Head from "next/head";

export default function Home() {
  const [showWidget, setShowWidget] = useState(false);
  const [referenceElement, setReferenceElement] = useState(null);
  const [popperElement, setPopperElement] = useState(null);
  const { styles, attributes } = usePopper(referenceElement, popperElement, {
    modifiers: [{ name: "offset", options: { offset: [10, 10] } }],
  });
  return (
    <div className="flex flex-col items-center justify-center min-h-screen py-2">
      <Head>
        <title>React Feedback Widget Example</title>
        <link rel="icon" href="/favicon.ico" />
      </Head>
      <button
        onClick={() => setShowWidget(!showWidget)}
        ref={setReferenceElement}
        className="absolute bottom-4 right-4 p-2 rounded-l-lg rounded-t-lg bg-blue-700 hover:bg-blue-600 shadow-md hover:shadow-lg text-white"
        type="button"
      >
        Feedback?
      </button>
      {showWidget && (
        <div
          className="bg-white p-2 rounded-md shadow-lg"
          ref={setPopperElement}
          style={styles.popper}
          {...attributes.popper}
        >
          <form>
            <label for="feedback">How can we improve?</label>
            <div className="flex flex-col space-y-2">
              <textarea
                id="feedback"
                className="p-2 h-24 w-64 border border-gray-400 rounded-md"
                type="textarea"
              ></textarea>
              <button
                className="p-2 ml-auto rounded-lg bg-blue-700 hover:bg-blue-600 shadow-md hover:shadow-lg text-white"
                type="submit"
              >
                Send Feedback
              </button>
            </div>
          </form>
        </div>
      )}
    </div>
  );
}

Github Repo