DEV Community: Andrew Jones

How to Use Function Calling with OpenAI's new Tools Feature to Solve Word Problems

Andrew Jones — Fri, 05 Jan 2024 21:45:22 +0000

With the launch of OpenAI's Assistants, the Function Calling system previously built into the openai SDK was deprecated and replaced with tools. The tools parameter allows developers to call OpenAI hosted tools, like file retrieval and code interpreter, and allows us to define custom tools that our own code handles, using function calling.

This allows us to add powerful functionality to our AI applications, because traditional code has different strengths from LLMs - such as mathematical operations, asynchronous requests for data, and more.

However, the new Tools system has some different nuances from the old method of Function Calling. This guide shows how to use function calling with the Tools system.

Goals

As a demonstration, we'll make an AI tool that can solve word problems. The flow will look like this:

We'll use the GPT API to understand the word problems and output the needed mathematical expression as a function call, as well as units for the answer. LLMs are great for this purpose due to semantic understanding of natural language.
Our code will call a mathematical expression solver (JS library) using the output from step 1.
We'll ask the LLM to generate a complete sentence to solve the problem, given the answer from step 2 and units from step #1 (remember your middle school math teacher docking points when you didn't answer a word problem with a complete sentence?)

Tutorial

First, setup a project:
mkdir word-problem-solver && cd word-problem-solver
npm init
npm i openai math-expression-evaluator

Create a file such as solve.js in your project folder, and set up the API client and expression solver:

const OpenAI = require("openai");
const MathExp = require("math-expression-evaluator");
const mathExp = new MathExp();

const openai = new OpenAI({
  apiKey: "sk-...", //replace with your API key
});

Next, we need to design our schema for function calling. I used functioncalling.ai, a tool I developed for this purpose, for this step - though this is entirely optional. You can run tests on sample word problems or even allow it to generate tests for you.

In the end, my schema looked like this:

[
  {
    type: "function",
    function: {
      name: "calculate",
      description:
        "Use math-expression-evaluator to simplify an expression to solve the word problem",
      parameters: {
        type: "object",
        properties: {
          expression: {
            type: "string",
            description:
              "the mathematical expression to simplify to solve the word problem. DO NOT include any units.",
          },
          answerUnit: {
            type: "string",
            description:
              "unit for the answer to the word problem, if applicable",
          },
        },
        required: ["expression", "answerUnit"],
      },
    },
  },
]

Paste that into your code as a constant:
const GPT_TOOLS = [...]

You could add additional tools by adding to the tools array, following OpenAI's docs for other types of tools, or adding more functions with similar schemas.

Next, we'll create a main async function and a messages array to track our conversation:

const main = async () => {
    const messages = [
    {
      role: "user",
      content: "13 candy bars weigh 26 oz. How much would 35 candy bars weigh?",
    },
  ]
}
main();

Our first call is where gpt-4 will pick the function to call (which must be calculate or none, since we only have one function), and output the expression to simplify, and the units for our answer. Make sure to push all messages to the messages array to maintain context throughout the conversation.

  const chatCompletionForFunction = await openai.chat.completions.create({
    messages,
    model: "gpt-4", // using gpt-4 for highest accuracy
    tools: GPT_TOOLS,
    tool_choice: "auto",
    temperature: 0.25,
  });

  messages.push(chatCompletionForFunction.choices[0].message);

Next we'll add handling for the output of the tool selection / function calling. We need to track the tool_call_id, which OpenAI uses to provide proper context for further messages - this is one of the key differences from the old function calling system.


  const tool_call_id =
    chatCompletionForFunction.choices[0].message.tool_calls[0].id;
  const function_to_call =
    chatCompletionForFunction.choices[0].message.tool_calls[0].function.name;

  if (function_to_call === "calculate") {
    const generatedParams = JSON.parse(
      chatCompletionForFunction.choices[0].message.tool_calls[0].function
        .arguments
    );

    const expression = generatedParams.expression;
    const answerUnit = generatedParams.answerUnit;
}

Finally, we'll actually simplify the expression, and use the LLM to write a people-friendly response:

    .....
    const expression = generatedParams.expression;
    const answerUnit = generatedParams.answerUnit;

    const lexed = mathExp.lex(expression);
    const postfixed = mathExp.toPostfix(lexed);
    const answer = mathExp.postfixEval(postfixed);

    // Add two messages, one to tell the LLM the output of the tool, and one to prompt it to generate a complete answer.
    messages.push(
      {
        role: "tool",
        tool_call_id,
        name: function_to_call.name,
        content: `${answer} ${answerUnit}`,
      },
      {
        role: "user",
        content: `Given the expression ${expression}, the answer is ${answer} ${answerUnit}. Write a complete sentence to answer the word problem.`,
      }
    );
    const chatCompletion = await openai.chat.completions.create({
      messages,
      model: "gpt-3.5-turbo-1106", // using GPT-3.5 for a lower cost since this is an "easy" step
      temperature: 0.4,
    });
    messages.push(chatCompletion.choices[0].message); // only needed if we intent to continue the conversation, but included here for completeness
    console.log(chatCompletion.choices[0].message.content);
}

Now we can run the code, and you should see output like: 35 candy bars would weigh 70 oz.

Voila! Next try different word problems in the prompt, or turn this into an interactive CLI with Node's readline, or try adding more functions, or try implementing function calling in your own projects!

Complete code

const OpenAI = require("openai");
const MathExp = require("math-expression-evaluator");
const mathExp = new MathExp();

const openai = new OpenAI({
  apiKey: "sk-...", // YOUR API KEY
});

const GPT_TOOLS = [
  {
    type: "function",
    function: {
      name: "calculate",
      description:
        "Use math-expression-evaluator to simplify an expression to solve the word problem",
      parameters: {
        type: "object",
        properties: {
          expression: {
            type: "string",
            description:
              "the mathematical expression to simplify to solve the word problem. DO NOT include any units.",
          },
          answerUnit: {
            type: "string",
            description:
              "unit for the answer to the word problem, if applicable",
          },
        },
        required: ["expression", "answerUnit"],
      },
    },
  },
];

const main = async () => {
  const messages = [
    {
      role: "user",
      content: "13 candy bars weigh 26 oz. How much would 35 candy bars weigh?",
    },
  ];
  const chatCompletionForFunction = await openai.chat.completions.create({
    messages,
    model: "gpt-4",
    tools: GPT_TOOLS,
    tool_choice: "auto",
    temperature: 0.25,
  });

  messages.push(chatCompletionForFunction.choices[0].message);

  const tool_call_id =
    chatCompletionForFunction.choices[0].message.tool_calls[0].id;
  const function_to_call =
    chatCompletionForFunction.choices[0].message.tool_calls[0].function.name;

  if (function_to_call === "calculate") {
    const generatedParams = JSON.parse(
      chatCompletionForFunction.choices[0].message.tool_calls[0].function
        .arguments
    );

    const expression = generatedParams.expression;
    const answerUnit = generatedParams.answerUnit;

    const lexed = mathExp.lex(expression);
    const postfixed = mathExp.toPostfix(lexed);
    const answer = mathExp.postfixEval(postfixed);

    messages.push(
      {
        role: "tool",
        tool_call_id,
        name: function_to_call.name,
        content: `${answer} ${answerUnit}`,
      },
      {
        role: "user",
        content: `Given the expression ${expression}, the answer is ${answer} ${answerUnit}. Write a complete sentence to answer the word problem.`,
      }
    );
    const chatCompletion = await openai.chat.completions.create({
      messages,
      model: "gpt-3.5-turbo-1106",
      temperature: 0.4,
    });
    messages.push(chatCompletion.choices[0].message); // not needed in this use case but included for completeness
    console.log(chatCompletion.choices[0].message.content);
  }
};

main();

How to Get Audio Transcriptions from Whisper without a File System

Andrew Jones — Sun, 10 Dec 2023 20:42:11 +0000

Whisper is OpenAI's intelligent speech-to-text transcription model. It allows developers to enter audio and an optional styling prompt, and get transcribed text in response.

However, the official OpenAI Node.js SDK API docs only show one way to use Whisper - reading an audio file with fs.

async function main() {
  const transcription = await openai.audio.transcriptions.create({
    file: fs.createReadStream("audio.mp3"),
    model: "whisper-1",
  });

  console.log(transcription.text);
}

That works fine if you have static files... but in any consumer application, we'll be processing data from an end-user client such as an app or web browser. To receive audio from thousands of users and save it as files is a major waste of disk space and a huge ineffeciency. Plus, serverless deployment is extremely popular today, and in a serverless environment we usually don't have persistent file storage. I wrote this article because it was surprisingly hard to figure out how to achieve audio transcription without saving the audio as a file first.

How to use Whisper without files

On the client-side, you'll need to get your audio into a Base64 encoded string. I'm using the library "@ricky0123/vad-react" for this purpose, which comes with utilities to accomplish that:

onSpeechEnd: (audio) => {
      const wavBuffer = utils.encodeWAV(audio);
      const base64 = utils.arrayBufferToBase64(wavBuffer);
      const audioUrlAsData = `${base64}`;
      // chose POST here with a payload to ensure the Base64 string doesn't violate the max length of a URL
      fetch("/api/transcribe", {
        method: "POST",
        body: JSON.stringify({ audioData: audioUrlAsData }), 
      })
}

Then on the server-side, the trick is to create a buffer from the base64 data and use the undocumented toFile function from OpenAI's library.

import OpenAI, { toFile } from "openai";

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

export default async function handler(
  req,
  res
) {
  try {
    // Extract Base64 encoded data from the request
    const bodyData = JSON.parse(req.body);
    const base64Audio = bodyData.audioData;

    // Decode Base64 to binary
    const audioBuffer = Buffer.from(base64Audio, "base64");

    // Use OpenAI API to transcribe the audio
    const transcription = await openai.audio.transcriptions.create({
      file: await toFile(audioBuffer, "audio.wav", {
        contentType: "audio/wav",
      }),
      model: "whisper-1",
    });

    // Send the transcription text as response
    res.json({ transcription: transcription.text });
  } catch (error) {
    console.error("Error during transcription:", error);
    res.status(500).send("Error during transcription");
  }
}

Voila! Through this process, you can use Whisper without saving audio from every user as static files, allowing it to be used in a serverless environment.

Why and How to Check the Server-side Rendered Content on your Website

Andrew Jones — Sun, 12 Nov 2023 19:18:06 +0000

So you've built a website powered by a JavaScript framework with server-side rendering, like Next.js, Gatsby, or Nuxt. However, you notice some issues with SEO - maybe Google isn't discovering links on your site properly, or your PageSpeed Insights performance score is hurt by high cumulative layout shift (CLS). Maybe your page feels jumpy as it loads.

These problems are common when developers write isomorphic code - code that runs both on the server and client. In our optimized development environment, with fast computers and internet speeds, the client-side rendering process of React and other JS frameworks is so fast that we don't often see what the page looks like and how it functions before and while the JavaScript executes.

Why does the server-side rendered UI matter if the page looks fine after it's done loading?

Most search engines either don't run client-side JavaScript at all, or defer it (like Google does), so we need to ensure all of the content that we can render on the server is rendering correctly. From the linked Google documentation page:

Keep in mind that server-side or pre-rendering is still a great idea because it makes your website faster for users and crawlers, and not all bots can run JavaScript.

Plus, a stable loading experience is important to build a usable, accessible experience for users in varying network and device conditions. If we our content jumps around as the site loads, it can be disorienting for users and they may tap on unintended elements. Jumpiness during loading raises your CLS score, hurting your performance score and hurting your Google Search ranking - so it's important to monitor how our page loads.

How can I check the Server-side Rendered Content?

I recommend anyone building a site with server-side and client-side rendering try this debugging method:

Disable JavaScript in Chrome by following these steps: https://developer.chrome.com/docs/devtools/javascript/disable/
Refresh your page and play around - confirm the content is in place, try clicking on different parts of the page, etc.

Some issues to check for which I've run into in the past:

Links not working, because they're actually buttons using JS to act like links, instead of actual <a> tags - which will prevent or delay pages from being discovered
Some content not rendering, because a code issue is causing the content to be rendered on the client-side only - which worsens UX due to content loading only after client-side JS takes over (hydration), and hurts your performance score due to content shifting during hydration instead of initially loading correctly - which also has a negative SEO impact. Additionally, client-side-only content loads later, so Google may deprioritize links/images or miss them entirely.

Personal, I also find this to be an interesting, educational exercise to see how your website acts in different conditions.

How do I fix these issues?

While the exact issue can vary, here are a few suggestions and things to check:

Make sure links are actually links. If you need them to look like a button, use an <a> tag styled as a button with CSS.
Make sure the data-based parts of the page are rendered correctly, to confirm your server-side data fetching is working.
If you fetch or revalidate any data on the client-side, make sure it's also being fetched in the SSR process and the client-side and server-side data is formatted the same/as expected.
Watch out for any superfluous "client-side-only" checks. For example, many JS frameworks allow you to use typeof window !== "undefined" to check when you're rendering on the server or the client. If you need to use that, make sure it's moved to the most granular level possible - server-side render whatever you can.
If there's an element you really can't server-side render, such as a customer's personal info on a static page, you should server-side render a placeholder to prevent CLS.

Note: most interactive features of your site probably won't work, unless you use native web forms/form handlers. That's expected, since we mostly build interactivity using JS. You want to focus on checking the content that should be server-side rendered.

Good luck!

A Better Guide To Forms in React

Andrew Jones — Wed, 11 Oct 2023 03:01:40 +0000

The React ecosystem is lush with libraries, articles, videos, and more resources for every web development topic you could imagine. However, over time, many of these resources become outdated with modern best practices.

Working on a complex dynamic form recently for an AI side project, and researching good guides to React forms, I realized that most of the resources about building forms are outdated, often by many years.

This article will explain modern best practices for building forms in React, how to build dynamic forms, how forms relate to React Server Components, and more. Finally, after gaining an understanding of those topics, I’ll conclude with an explanation of what I found lacking in other guides, and recommendations from my experience with React.

Controlled vs Uncontrolled

The key to understanding React forms is the idea of “controlled” vs “uncontrolled” inputs, which are two different methods of building forms in React.

Controlled forms store each input’s value in React state, and then set the value of each input on each re-render from that state. If another function updates the state, that will immediately be reflected in the components’ values. If your React logic does not render the form inputs, the values will still be maintained in your app’s state. Controlled inputs tend to give you more options in your React logic, such as complex, non-HTML-standard form validation as the user types (maybe checking password strength), code manipulation of the values of the inputs (such as hyphenating phone numbers as the user types).

They look something like this:

import React, { useState } from 'react';

function ControlledForm() {
  const [value, setValue] = useState('');

  const handleChange = (event) => {
    setValue(event.target.value);
  };

  const handleSubmit = () => {
    // no "submit event" param needed above ^
    sendInputValueToApi(value).then(() => /* Do something */)
  };

  return (
    <>
      <input type="text" value={value} onChange={handleChange} />   
      <button onClick={handleSubmit}>Send</button>
    </>
}

Note that it would be more semantically correct to wrap the input in a <form>, and give the input a name, but it’s not functionally needed because the data is already saved in a state, so we don’t need a real onSubmit event and our input doesn’t need to be accessed directly when the button is pressed.

There are a few downsides:

You may not want to re-render the form every time the user types.
You need to write a lot of code to manage complex forms, because you wind up with a lot of states as your form grows in size. That leads to boilerplate useState and state-setting, which expands your code.
It’s harder to build dynamic forms where the number of fields is variable, because you can’t conditionally use hooks like useState. To fix this problem: (a) Your entire form state likely becomes a single, huge object, (b) Then all children components will re-render on every input change due to using an object as a state rather than primitives. (c) The only way around this, memoization, adds a lot more boilerplate.

This can lead to performance challenges with bigger forms like spreadsheets, tables, etc.

import React, { useState } from "react";

function CumbersomeForm() {
  const [formData, setFormData] = useState({
    firstName: "",
    lastName: "",
    email: "",
    address: "",
    // ... potentially many more individual properties
  });

  const handleChange = (e) => {
    const { name, value } = e.target;
    setFormData((prevState) => ({ ...prevState, [name]: value }));
  };

  return (
    <>
      <label>First Name:</label>
      <input
        type="text"
        name="firstName"
        value={formData.firstName}
        onChange={handleChange}
      />
      <label>Last Name:</label>
      <input
        type="text"
        name="lastName"
        value={formData.lastName}
        onChange={handleChange}
      />
      <label>Email:</label>
      <input
        type="email"
        name="email"
        value={formData.email}
        onChange={handleChange}
      />
      <label>Address:</label>
      <input
        type="text"
        name="address"
        value={formData.address}
        onChange={handleChange}
      />
      {/* ... potentially many more individual input fields */}
    </>
  );
}

On the other hand, uncontrolled inputs do not save the input values in React state. Instead, uncontrolled forms use the native, built-in <form> functionalities of vanilla HTML and JavaScript to manage data. So, for example, instead of setting a state with the input’s value every time it’s changed, and then setting its value prop, the browser manages the input value. Our React component never needs or uses the values - when the component is rendered, React will add the onsubmit event listener to the form, and when the submit button is pushed, our handleSubmit function runs. Instead of using React state, it’s much closer to the way a plain HTML form would work without any JavaScript.

function UncontrolledForm() {

  const handleSubmit = (event) => {
    event.preventDefault();

    const formData = new FormData(event.target);
    const inputValue = formData.get('inputName');

    sendInputValueToApi(inputValue).then(() => /* Do something */)  
  };

  return (
    <form onSubmit={handleSubmit}>
      <input type="text" name="inputName" />
      <button type="submit">Send</button>
    </form>
  );
}

A benefit of this approach is that there’s less boilerplate. Compare these two inputs:

// Controlled

const [value, setValue] = useState('');

const handleChange = (event) => {
    setValue(event.target.value);
};
...
<input type="text" value={value} onChange={handleChange} />

// Uncontrolled

<input type="text" name="inputName" />

The difference is significant even with 1 input, but becomes even more clear with many more inputs. Compare the longer controlled form above with this one:

import React from "react";

function UncontrolledForm() {
  const handleSubmit = (event) => {
    event.preventDefault();
    const formData = new FormData(event.target);
    for (let [key, value] of formData.entries()) {
      console.log(`${key}: ${value}`);
    }
  };

  return (
    <form onSubmit={handleSubmit}>
      <label>First Name:</label>
      <input type="text" name="firstName" />

      <label>Last Name:</label>
      <input type="text" name="lastName" />

      <label>Email:</label>
      <input type="email" name="email" />

      <label>Address:</label>
      <input type="text" name="address" />
      {/* ... potentially many more individual input fields */}
      <button type="submit">Submit</button>
    </form>
  );
}

This form has less boilerplate than the controlled version, and we don’t need to manage multiple hook calls or a large complex state object. In fact, there is no state here at all. This form could have hundreds or thousands of children, and they will never cause each other to re-render. With this method, forms perform better, have less boilerplate, and are easier to reason about.

The downside of uncontrolled forms is that you don’t have direct access to each input’s value. This can make custom validation more difficult, and makes direct input value manipulation impossible.

Caveats and Compromises

(Don't) useRef

Many articles recommend using a ref on each input in uncontrolled forms instead of using new FormData() which I believe is because the FormData API is lesser-known. However, it’s been a standard for almost a decade and has been supported by all major browsers for many years. I strongly recommend that you do not use a useRef call for each form input, as this introduces some of the same challenges and boilerplate as useState calls.

However, there are some use-cases where a ref can help.

Focus Management - programmatically focus on form elements, especially when dealing with dynamic form fields.

function MyForm() {
  const inputRef = useRef(null);

  const focusInput = () => {
    inputRef.current.focus();
  };

  return (
    <form>
      <input ref={inputRef} type="text" />
      <button type="button" onClick={focusInput}>
        Focus Input
      </button>
    </form>
  );
}

Calling Methods on Child Components’ HTML Elements - for example, if your form is broken into subcomponents and you need to focus on one of them from your parent form, or a sibling of the subcomponent, you need to pass around a ref from the parent, like this:

const ChildComponent = React.forwardRef((props, ref) => (
  <input ref={ref} type="text" />
));

function MyForm() {
  const inputRef = useRef(null);

  const focusInput = () => {
    inputRef.current.focus();
  };

  return (
    <form>
      <ChildComponent ref={inputRef} />
      <button type="button" onClick={focusInput}>
        Focus Input
      </button>
    </form>
  );
}

Other, non-form-specific ref use-cases - such as saving previous values with a useEffect, or measuring the physical size of an element like a textarea that a user can expand

Mixing Controlled and Uncontrolled

In many cases, you may need to control one or a few inputs. Manipulating a phone number as the user types it in is a great example. In those cases, even if you’re using an uncontrolled form, you can just use one controlled input. In that case, don’t use the state to access the input’s value in form submission - continue using new FormData(...) - just use the state to manage the display of the input in question.

function MixedForm() {
  const [phoneNumber, setPhoneNumber] = useState("");

  const handlePhoneNumberChange = (event) => {
    // Some function to format the phone number
    const formattedNumber = formatPhoneNumber(event.target.value);
    setPhoneNumber(formattedNumber);
  };

  const handleSubmit = (event) => {
    event.preventDefault();
    const formData = new FormData(event.target);
    for (let [key, value] of formData.entries()) {
      console.log(`${key}: ${value}`);
    }
  };

  return (
    <form onSubmit={handleSubmit}>
      <label>Name:</label>
      <input type="text" name="name" />

      <label>Email:</label>
      <input type="email" name="email" />

      <label>Phone Number:</label>
      <input
        type="tel"
        name="phoneNumber"
        value={phoneNumber}
        onChange={handlePhoneNumberChange}
      />

      <label>Address:</label>
      <input type="text" name="address" />

      <button type="submit">Submit</button>
    </form>
  );
}

// Helper function for phone number formatting (just an example)
function formatPhoneNumber(number) {
  // Format the phone number as desired. This is just a basic example.
  return number.replace(/\D/g, "").slice(0, 10);
}

Note: minimize states. In this example, you wouldn’t want a useState for saving the raw phone number, and a separate useState for saving the formatted phone number, with an effect to sync them - that would cause unnecessary re-renders.

Speaking of optimizing re-renders, we can also move the controlled input to its own component to minimize re-renders of the rest of the form.

const PhoneInput = () => {
  const [phoneNumber, setPhoneNumber] = useState("");

  const handlePhoneNumberChange = (event) => {
    // Some function to format the phone number
    const formattedNumber = formatPhoneNumber(event.target.value);
    setPhoneNumber(formattedNumber);
  };

  return (
    <input
      type="tel"
      name="phoneNumber"
      value={phoneNumber}
      onChange={handlePhoneNumberChange}
    />
  );
};

function MixedForm() {
  const handleSubmit = (event) => {
    event.preventDefault();
    const formData = new FormData(event.target);
    for (let [key, value] of formData.entries()) {
      console.log(`${key}: ${value}`);
    }
  };

  return (
    <form onSubmit={handleSubmit}>
      <label>Name:</label>
      <input type="text" name="name" />

      <label>Email:</label>
      <input type="email" name="email" />

      <label>Phone Number:</label>
      <PhoneInput />

      <label>Address:</label>
      <input type="text" name="address" />

      <button type="submit">Submit</button>
    </form>
  );
}

// Helper function for phone number formatting (just an example)
function formatPhoneNumber(number) {
  // Format the phone number as desired. This is just a basic example.
  return number.replace(/\D/g, "").slice(0, 10);
}

If you’re used to controlled inputs, then on a first glance at the code above, you might think, “how does the parent form know the value of the child phone input if it’s not passing in some state setter or a ref?” To understand that, remember that when the React code is rendered as HTML, the browser will only “see” the HTML for the form and the inputs inside, including the input which the PhoneInput renders - the structure of our React components has no impact on the functionality of our rendered HTML. Then, that input value will be included in the FormData like any other field in the form. This is the power of encapsulation and composition of components in React. We can localize the re-renders to the minimum-affected JSX, while the DOM all still comes together as native HTML.

This is also useful for building dynamic forms. If you were. managing a state object using controlled components, you'd need to add and remove properties manually when form fields were created or removed. On the other hand, with uncontrolled components, you can just render/unrender inputs with unique names, and their values will automatically appear in the FormData on-submit.

But wait… how do I validate input values in uncontrolled inputs?

If you thought of this, you’re not alone! When validation is required before submitting a form, React devs tend to jump to using controlled components with state.

Many developers don’t realize that you don’t always need React or custom JavaScript for these types of validations - in fact, the web platform has them built in as attributes on form inputs! Check out this page on MDN for the details: https://developer.mozilla.org/en-US/docs/Learn/Forms/Form_validation

Without any JS, you can require input values, set length requirements, and set format requirements using regular expressions (AI tools like ChatGPT are great for helping with complex Regex)

Error Handling

On a related topic, devs tend to reach for controlled components when we need to show some validation errors on the client-side. However, instead of using controlled components and setting values on every input change, when the requirements allow it I would prefer to use uncontrolled components and manage error handling in my onSubmit function. This minimizes the amount of state and state updates in your forms. Maybe it looks like this:

function UncontrolledForm() {
  const [errors, setErrors] = useState({});

  const handleSubmit = (event) => {
    event.preventDefault();
    const formData = new FormData(event.target);

    let validationErrors = {};

    // Custom validation: Ensure the email domain is "example.com"
    const email = formData.get("email");
    if (email && !email.endsWith("@example.com")) {
      validationErrors.email = "Email must be from the domain example.com.";
    }

    if (formData.get("phoneNumber").length !== 10) {
      validationErrors.phoneNumber = "Phone number must be 10 digits.";
    }

    if (Object.keys(validationErrors).length > 0) {
      setErrors(validationErrors);
    } else {
      // Handle the successful form submission, e.g., sending formData to a server
      console.log(Array.from(formData.entries()));
      setErrors({}); // Clear any previous errors
    }
  };

  return (
    <form onSubmit={handleSubmit}>
      <label>Name:</label>
      <input type="text" name="name" required />
      {errors.name && <div className="error">{errors.name}</div>}

      <label>Email (must be @example.com):</label>
      <input type="email" name="email" required />
      {errors.email && <div className="error">{errors.email}</div>}

      <label>Phone Number (10 digits):</label>
      <input type="tel" name="phoneNumber" required pattern="\d{10}" />
      {errors.phoneNumber && <div className="error">{errors.phoneNumber}</div>}

      <button type="submit">Submit</button>
    </form>
  );
}

export default UncontrolledForm;

There’s one more big benefit to using uncontrolled components.

Forms in React Server Components

React Server Components (RSC) uses server-side frameworks to render some of your components and thereby reduce the amount of JavaScript that web browsers download to run your website. This can drastically improve the performance of your website.

RSC has a big impact on the way we write forms, because, for the first time, if our form inputs do not use state, they can be rendered to HTML on the server and ship 0 JavaScript to the browser. This means that uncontrolled forms can be interactive without JavaScript, making them function earlier rather than waiting for JS to download and execute - in effect, it makes your website feel snappier, there’s no delay to interactivity.

Using Next.js, you can also use Server Actions in your forms, so you don’t need to write an API for your form to interact with - all you need is a handler function. You can find out more info about this topic from the Next.js docs or Lee’s excellent explainer video here.

If you build a mixed form with some controlled inputs in an RSC-rendered form, make sure to encapsulate your controlled components into a separate client-component, similar to the <PhoneInput /> component from earlier, but in its own file. This ensures that only the minimum amount of JavaScript needed will be bundled. It may look like this:

// page.jsx
import { PhoneInput } from "./PhoneInput";

export default function Page() {
  async function create(formData: FormData) {
    "use server";

    // ... use the FormData
  }

  return (
    <form action={create}>
      <label>Name:</label>
      <input type="text" name="name" />

      <label>Email:</label>
      <input type="email" name="email" />

      <label>Phone Number:</label>
      <PhoneInput />

      <label>Address:</label>
      <input type="text" name="address" />

      <button type="submit">Submit</button>
    </form>
  );
}

// PhoneInput.jsx
"use client";

// Helper function for phone number formatting (just an example)
function formatPhoneNumber(number) {
  // Format the phone number as desired. This is just a basic example.
  return number.replace(/\D/g, "").slice(0, 10);
}

import { useState } from "react";

export const PhoneInput = () => {
  const handlePhoneNumberChange = (event) => {
    // Some function to format the phone number
    const formattedNumber = formatPhoneNumber(event.target.value);
    setPhoneNumber(formattedNumber);
  };
  const [phoneNumber, setPhoneNumber] = useState("");

  return (
    <input
      type="tel"
      name="phoneNumber"
      value={phoneNumber}
      onChange={handlePhoneNumberChange}
    />
  );
};

Form Libraries

There are many great form libraries in the React ecosystem, designed for controlled inputs. Recently I’ve been using React Hook Form for those applications, though I’ve been leaning more heavily on uncontrolled inputs so I don’t need a library to manage form state. (Edit: some popular options are React Hook Form, Formik, and Informed)

Conclusions, Comparisons, and Recommendations

This article was motivated by the confusing, out-dated, and/or misleading articles that show up near the top of the Google search results for “react forms.”

One of those articles says “the more idiomatic React way would be using controlled forms” - I don’t think either controlled or uncontrolled inputs are more “React idiomatic.” In reality, both types have their place, as discussed above. In fact, many older articles seem to recommend controlled inputs, for similarly vague or misleading reasons. I would argue that using uncontrolled inputs is actually more “web idiomatic” by nature of relying on less JavaScript, which allows uncontrolled inputs to shine in React Server Components.
Not one of the top articles uses FormData. For uncontrolled forms, at least two recommend multiple useRef calls, making your code inflexible and more boilerplatey.
Some of the top articles still use class components with no mention of functional components.

Some thoughts in conclusion:

In my experience, most forms are a mixture of uncontrolled and controlled inputs. The reason we have both of those options is flexibility and we shouldn’t be dogmatic. We can use them both together, as in the React Server Components example above (although server components are not needed for that example).
Today, I favor uncontrolled inputs wherever I can. I believe this simplifies the code structure and performance.
Use “new FormData(...)” in your onSubmit functions instead of using refs. Seriously.
Use composition and encapsulation for controlled inputs to minimize the impact of state updates on other components, and rely on the rendered, composed DOM for submit events.

I hope this article helps someone! Feel free to leave questions in the comments.

The Ultimate Guide to internationalization (i18n) in Next.js 13

Andrew Jones — Mon, 16 Jan 2023 19:19:56 +0000

Introduction

Internationalization can be a complex challenge for developers to get right, since it ties in with infrastructure. This post will teach you how to nail internationalized routing and content translations in your Next.js 13 website or web application.

I work in the e-commerce space, with brands spanning countries and continents - almost all of them require different website content per language or region. According to Vercel, the company behind Next.js:

72% of consumers are more likely to stay on your site if it's been translated and 55% of consumers said they only buy from e-commerce sites in their native language.
(quote source)

So it's incredibly important for your sites to have i18n support, to maximize users and conversion rates.

Understanding that, it may surprise you to find out that the new Next.js 13 app directory drops the i18n support the pages directory has had built-in since 2020.

Note: this tutorial assumes basic familiarity with the app directory in Next.js 13. More info linked at the bottom. Additionally, if you get lost, check out the finished code here.

How will I localize my Next.js 13 project without built-in support?

The built-in internationalized routing that launched in Next.js 10 in Oct. 2020 had many limitations. For example, the routing structure was opinionated, requiring a single URL subpath or different TLD, like example.com/en-US, example.com/en-CA or example.us, example.ca. URLs with two subpaths was not supported, so example.com/en/us would be impossible.

A year later, in Oct 2021, Next.js released with support for custom middleware, which allows developers to use our own business logic for routing. With this system, internationalized routing doesn't need to be built-in to the platform, because it's now entirely in our control how Next will route users. Since we control routing, and the file structure, we can now set up more flexible URL structures.

In Next.js 13, while the old pages directory maintains existing support for i18n routing, it's intentionally removed from app directory routing to put the power into developers' hands.

Goals

In this tutorial, we'll do something we couldn't do before Next.js 13 - set up international routing and translations using two subpaths like example.com/en/us. We will include static generation, locale detection, and handling a default locale which shouldn't require the subpath.

Our website will have 3 supported locales corresponding to the following regional dialects:

en-US which should be our default and have no URL subpath,
en-CA for Canadian English,
fr-CA for Canadian French

Set up the File Structure

Start by creating a Next.js app with the --experimental-app flag (as of Jan. 2023). We'll use TypeScript too.

Copy all of the content inside app/ into a new folder, app/[lang]/[country]. Also create a new file at the project-root level called middleware.ts. Your folder structure should look like this:

In middleware.tsx, add an import of the Next.js Request type and add an empty default function and a matcher to ignore non-content URLs:

export function middleware(request: NextRequest) {

}

export const config = {
// do not localize next.js paths
  matcher: ["/((?!api|_next/static|_next/image|assets|favicon.ico|sw.js).*)",],
};

In page.tsx replace the default function with the following, which will display the parameters the page is receiving:

export default function Home({
  params,
}: {
  params: { lang: string; country: string };
}) {
  return <h1>{JSON.stringify(params)}</h1>;
}

Now you should be able to run your project with npm run dev and then navigate to any URL with two parameters, such as localhost:3000/hello/world, and see something like this:

The URL parameters are working great, but this isn't a valid locale!

Handle Locale Validation, Redirects, and Rewrites with Middleware

We'll use middleware to validate that the user is accessing a valid locale, at the correct URLs, and display the default locale when a user is at the site root with no subpath.

The middleware runs on every request to the site that matches the regex we added above. So, we can use it to redirect the user, or use a "rewrite" which allows us to show one page's content even when the user is on a different URL.

First, let's specify our default locale, list of allowed locales, and a convenience function to help us process the locale parts. (Note: you wouldn't need the convenience function if you only use one subpath, like example.com/en-US, but we'll use the more complex case in this guide.)

// middleware.ts

import { NextRequest, NextResponse } from "next/server";

const defaultLocale = "en-US";
let locales = ["en-US", "en-CA", "fr-CA"];

type PathnameLocale = {
  pathname: string;
  locale?: never;
};
type ISOLocale = {
  pathname?: never;
  locale: string;
};

type LocaleSource = PathnameLocale | ISOLocale;

const getLocalePartsFrom = ({ pathname, locale }: LocaleSource) => {
  if (locale) {
    const localeParts = locale.toLowerCase().split("-");
    return {
      lang: localeParts[0],
      country: localeParts[1],
    };
  } else {
    const pathnameParts = pathname!.toLowerCase().split("/");
    return {
      lang: pathnameParts[1],
      country: pathnameParts[2],
    };
  }
};

export function middleware(request: NextRequest) {

}

Next, let's figure out how to send users from /en/us, our default locale, back to /. First we'll check if the current requested URL path matches the default locale, and if it does then we'll remove the subpath.

export function middleware(request: NextRequest) {
  const pathname = request.nextUrl.pathname;

  const defaultLocaleParts = getLocalePartsFrom({ locale: defaultLocale });
  const currentPathnameParts = getLocalePartsFrom({ pathname });

  // Check if the default locale is in the pathname
  if (
    currentPathnameParts.lang === defaultLocaleParts.lang &&
    currentPathnameParts.country === defaultLocaleParts.country
  ) {
    // we want to REMOVE the default locale from the pathname,
    // and later use a rewrite so that Next will still match
    // the correct code file as if there was a locale in the pathname
    return NextResponse.redirect(
      new URL(
        pathname.replace(
          `/${defaultLocaleParts.lang}/${defaultLocaleParts.country}`,
          pathname.startsWith("/") ? "/" : ""
        ),
        request.url
      )
    );
  }
}

With this change, try navigating to localhost:3000/en/us and you should be redirected to localhost:3000. If you try navigating to localhost:3000/en/ca, you should not be redirected.

Now that we're redirecting users from localhost:3000/en/us and invalid locales to localhost:3000, we need Next.js to display / as if it matched the filepath /app/[lang]/[country] to prevent an incorrect 404, since we don't have a root app/page.tsx file. We also should serve a 404 if the pathname doesn't match any valid locale. We can kill two birds with one stone using the following loop and rewrite:

const pathnameIsMissingValidLocale = locales.every((locale) => {
    const localeParts = getLocalePartsFrom({ locale });
    return !pathname.startsWith(`/${localeParts.lang}/${localeParts.country}`);
  });

  if (pathnameIsMissingValidLocale) {
    // rewrite it so next.js will render `/` as if it was `/en/us` 
    return NextResponse.rewrite(
      new URL(
        `/${defaultLocaleParts.lang}/${defaultLocaleParts.country}${pathname}`,
        request.url
      )
    );
  }

How does this work? At this point, if you try to access localhost:3000/hello/world, since it doesn't match any locale in our array, Next.js will treat it as localhost:3000/en/us/hello/world, a file path which doesn't exist, and give you a 404 page.

With that, we've achieved our routing requirements! 🎉

Automatic locale detection

Now that we have a great routing setup, we can optimize the user experience by automatically redirecting users to the correct route based on their location and language preferences. This way they shouldn't need to manually use a language picker on your site (though it may still be a feature you want to make available).

First, run npm i accept-language-parser and npm i @types/accept-language-parser --save-dev. accept-language-parser is a small library that will help us process the request headers to find the best language for a user.

Next, add this import to the top of your middleware.ts file, and add the findBestMatchingLocale function under the locales declarations:

import langParser from "accept-language-parser";

const defaultLocale = "en-US";
const locales = ["en-US", "en-CA", "fr-CA"];

const findBestMatchingLocale = (acceptLangHeader: string) => {
  // parse the locales acceptable in the header, and sort them by priority (q)
  const parsedLangs = langParser.parse(acceptLangHeader);

  // find the first locale that matches a locale in our list
  for (let i = 0; i < parsedLangs.length; i++) {
    const parsedLang = parsedLangs[i];
    // attempt to match both the language and the country
    const matchedLocale = locales.find((locale) => {
      const localeParts = getLocalePartsFrom({ locale });
      return (
        parsedLang.code === localeParts.lang &&
        parsedLang.region === localeParts.country
      );
    });
    if (matchedLocale) {
      return matchedLocale;
    }
    // if we didn't find a match for both language and country, try just the language
    else {
      const matchedLanguage = locales.find((locale) => {
        const localeParts = getLocalePartsFrom({ locale });
        return parsedLang.code === localeParts.lang;
      });
      if (matchedLanguage) {
        return matchedLanguage;
      }
    }
  }
  // if we didn't find a match, return the default locale
  return defaultLocale;
};

This function will find the best matching locale by iterating through the priority-sorted locales from the request and comparing them to the available locales on our site. First it will try to find an available locale matching the requested language and country, and if none is found then it will fall back to just matching the language, and finally fall back to the default locale.

In the last step, we rewrote all URLs without a matching locale subpath to use the en-US locale. In this step, we'll redirect the user if there's a matching locale other than the default. If the default is the best match, we'll use the same rewrite as before.

Update the final if-statement in the middleware file to the following:

  if (pathnameIsMissingValidLocale) {
    // rewrite it so next.js will render `/` as if it was `/en/us`

    const matchedLocale = findBestMatchingLocale(
      request.headers.get("Accept-Language") || defaultLocale
    );

    if (matchedLocale !== defaultLocale) {
      const matchedLocaleParts = getLocalePartsFrom({ locale: matchedLocale });
      return NextResponse.redirect(
        new URL(
          `/${matchedLocaleParts.lang}/${matchedLocaleParts.country}${pathname}`,
          request.url
        )
      );
    } else {
      return NextResponse.rewrite(
        new URL(
          `/${defaultLocaleParts.lang}/${defaultLocaleParts.country}${pathname}`,
          request.url
        )
      );
    }
  }

How do we test this? Chrome allows us to simulate locales from different requests.

In Chrome dev tools, press cmd + shift + P and type in sensors

Select "Show sensors" to open the menu, and add one of our supported locales:

Now if you attempt to navigate to the homepage without a sub-domain, you should be redirected to /fr/ca. Even if you enter fr-FR (French standard, rather than Canadian French), you'll still be redirectedto the French Canadian subpath instead of the English subpath. Voila! 🎊

Content Translation

For content translation, we'll use the official Next.js 13 guide to localization as a foundation, but add variable interpolation.

Start by moving all of our locale code out of middleware.ts into a new file also at the project root, i18n.ts:

// i18n.ts

export const defaultLocale = "en-US";
export const locales = ["en-US", "en-CA", "fr-CA"] as const;
export type ValidLocale = typeof locales[number];

type PathnameLocale = {
  pathname: string;
  locale?: never;
};
type ISOLocale = {
  pathname?: never;
  locale: string;
};

type LocaleSource = PathnameLocale | ISOLocale;

export const getLocalePartsFrom = ({ pathname, locale }: LocaleSource) => {
  if (locale) {
    const localeParts = locale.toLowerCase().split("-");
    return {
      lang: localeParts[0],
      country: localeParts[1],
    };
  } else {
    const pathnameParts = pathname!.toLowerCase().split("/");
    return {
      lang: pathnameParts[1],
      country: pathnameParts[2],
    };
  }
};

I also added the as const directive to add a ValidLocale enum based on the locales array. Make sure to update the imports in your middleware.tsx to reflect these changes.

Next, add a dictionaries folder in your project root with three JSON files, one for each locale:

(dictionaries/en-CA.json)
{
    "welcome": {
      "helloWorld": "Hello World, eh?",
      "happyYear": "Happy {{ year }}!"
    }
  }

(dictionaries/en-US.json)
{
  "welcome": {
    "helloWorld": "Hello World!",
    "happyYear": "Happy {{ year }}!"
  }
}

(dictionaries/fr-CA.json)
{
  "welcome": {
    "helloWorld": "Salut le Monde!",
    "happyYear": "Bonne année {{ year }}!"
  }
}

Next we'll add a translation function into i18n.ts which can be called by server components to import the dictionary and get the requested translation. This function will also handle variable interpolation similar to i18next

// i18n.ts
const dictionaries: Record<ValidLocale, any> = {
  "en-US": () =>
    import("dictionaries/en-US.json").then((module) => module.default),
  "en-CA": () =>
    import("dictionaries/en-CA.json").then((module) => module.default),
  "fr-CA": () =>
    import("dictionaries/fr-CA.json").then((module) => module.default),
} as const;

export const getTranslator = async (locale: ValidLocale) => {
  const dictionary = await dictionaries[locale]();
  return (key: string, params?: { [key: string]: string | number }) => {
    let translation = key
      .split(".")
      .reduce((obj, key) => obj && obj[key], dictionary);
    if (!translation) {
      return key;
    }
    if (params && Object.entries(params).length) {
      Object.entries(params).forEach(([key, value]) => {
        translation = translation!.replace(`{{ ${key} }}`, String(value));
      });
    }
    return translation;
  };
};

We can now use the translations in our page.tsx file. Since the function to import the translation files is async, we'll need to make the Home() function async as well:

// page.tsx

import { getLocalePartsFrom, locales, ValidLocale, getTranslator } from "@/i18n";

export default async function Home({
  params,
}: {
  params: { lang: string; country: string };
}) {
  const translate = await getTranslator(
    `${params.lang}-${params.country.toUpperCase()}` as ValidLocale // our middleware ensures this is valid
  );
  return (
    <div>
      <h1>{translate("welcome.helloWorld")}</h1>
      <h2>
        {translate("welcome.happyYear", {
          year: new Date().getFullYear(),
        })}
      </h2>
    </div>
  );
}

The translate function is created based off of the current locale subpath the user is on, and then content can be translated by adding keys to our JSON files. If there are any keys missing translations, the key itself will be rendered. And what's best - since this is all server-side code, we don't need to ship a huge JSON translations object to the browser.

How do I translate content in client components?

We're able to make our Home() function above asynchronous, and use getTranslator, because Home() is a server component.

We can't make client components asynchronous, so this is what I would recommend:

Maximize usage of server components.
Don't forget that you can render a server component as a child of a client component (see here for info). This means you can often break out static text elements into a separate server component.
For text that you unavoidably need to translate in a client component, you can execute the translations on the server component and pass the text into the client component as a prop.

If #3 concerns you, thinking about cases of prop-drilling, consider point #2 again. It should be rare to have deeply-nested child components that don't have a server-component parent or grandparent.

Use generateStaticParams to statically generate the localized pages and prevent build errors

If you try to build the project after the localization steps above, you should get an error, because Next will attempt to import a dictionary for the generic case of any string locales.

To fix this, we need to tell Next to only statically generate pages for our valid locales.

Add this function in page.tsx:

import { ....... } from "@/i18n";

export async function generateStaticParams() {
  return locales.map((locale) => getLocalePartsFrom({ locale }));
}

export default async function Home({
.......

Run yarn build to confirm everything is working, and the project should build! ✅

Then run yarn start and navigate to one of the localized subpaths to see the translations.

So far, I've had to include the same snippet on every page I create. For example:

// /app/[lang]/[country]/examplePage/page.tsx
import { getTranslator } from "@/i18n";
import { getLocalePartsFrom, locales, ValidLocale } from "@/i18n";

export async function generateStaticParams() {
  return locales.map((locale) => getLocalePartsFrom({ locale }));
}

export default async function ExamplePage({
  params,
}: {
  params: { lang: string; country: string };
}) {
  const translate = await getTranslator(
    `${params.lang}-${params.country.toUpperCase()}` as ValidLocale // our middleware ensures this is valid
  );
  return (
    <div>
      <h1>Example page: {translate("welcome.helloWorld")}</h1>
    </div>
  );
}

Taking it further

Ideas to consider:

You could test out different URL structures, such as single-subpath locales or different domains, by customizing the routing rules in the middleware.
You may want to customize the translation rules for missing keys. For example, instead of returning the key, return the default locale's text for that key.
You may want to have some pages unlocalized, which you can add in /app
You could build a frontend language picker component.
Make the dictionaries object dynamic based on the allowed locales

Conclusion!

The app directory in Next.js 13 doesn't include the built-in localization system of the pages directory.
Instead, it gives us more flexibility via middleware and powerful server-side rendering.
We used middleware logic to redirect users and rewrite pages based on the requested URL and each user-requests's Accept-Language header.
We used the power of server components to translate content and reviewed methods of translating client component content
We used generateStaticParams to correctly build the project

Thanks for reading, hope this helps! If you find anything wrong, please let me know in the comments or on Twitter

A big thanks to everyone who has contributed ideas on i18n in this GitHub issue

Next.js 13 App directory docs
Next.js 13 guide to localization

How to add User Accounts and Paid Subscriptions to your Next.js Website

Andrew Jones — Fri, 28 Jan 2022 19:01:35 +0000

Recently I spent a weekend adding paid subscriptions to my project, so I'm writing this article to share the process and what I wish I had known before I started!

This tutorial will assume some experience with React and TypeScript. You'll also need a database to use, any one that's compatible with Prisma. I'll be using Postgres. You can follow my last tutorial (up to the REST API section) for a beginner's guide on setting up Postgres locally, and an intro to Prisma.

The Goal

In this tutorial, we'll create a Next.js site, set up a database, add user registration (via GitHub OAuth), and give customers the ability to sign up for a paid subscription using a Stripe Checkout hosted page. Many of the same concepts apply even if you're using a different OAuth provider, a custom payment form, or a different payment provider.

We're going to set up a system like this: when a user registers for an account on your site, we'll also create a customer in Stripe's system for the user, and we'll save the Stripe customer ID in our database with the user's data. Then, when a user wants to add a subscription to their account on our site, we can use that Stripe customer ID to easily mark the user as a paying user in our system, and then allow them access to our services. We'll also discuss next steps to allow users to cancel their subscriptions and more. The flow will look like this:

TLDR

Set up a Next.js Project
Add Prisma and set up a Database
Add Next-Auth and configure account creation
Create a Stripe Account
On account-creation, use a Next-Auth event to create a Stripe Customer and link them
Allow the frontend to request a Stripe Payment Link from the backend, pre-linked to their customer ID
Use Stripe Webhooks to activate the customer's subscription in our database when they complete a checkout
Test the flow

Set Up a Project

Follow the excellent official guide here to set up a Next.js project. I'm using TypeScript, which works especially well with Prisma.



npx create-next-app@latest --typescript

When that's finished, make sure you have typescript and React types installed using:



npm install --save-dev typescript @types/react

To do some cleanup, you can delete all of the content inside the <main>...</main> section of index.tsx.

Adding Prisma and Database Setup

One mistake I made was that I implemented my entire authentication system and database schema without accounting for payment-related fields. We'll fix that by creating our initial schema with both next-auth and stripe in mind.

Next-Auth and Stripe

Next-Auth is a great way of easily adding user registration and authentication to your Next.js projects. Its docs provide you everything you need to get started with a huge variety of authentication providers and databases. You can read more about it at https://next-auth.js.org/.

Stripe is one of the most popular payment systems existing today. It essentially allows you to build payment forms into your apps, websites, and servers, and it handles all of the complex logic behind communicating with credit card companies and banks to actually get you your payment. It supports a ton of use cases, including paid subscriptions, which is what we'll use it for. Read more about it at https://stripe.com/.

Setting up the Prisma Schema

First, we'll set up Prisma. If you get stuck on this part, check Prisma's documentation. Start by creating a new folder in your project folder called prisma, and then a file called schema.prisma inside the folder.

Next we need to determine what else to put in this schema file. The schema file determines the structure of the database and TypeScript types that Prisma will generate.

We need to connect Next-Auth to Prisma, so that it can actually save user accounts after they're created. To do that, we'll use the official Next-Auth Prisma Adapter. We'll install it later, but for now, copy the text from the schema file shown here and paste it into your schema file. These are the fields which the Next-Auth Prisma Adapter requires for its features to work. If you're not using Postgres, you'll need to change the database part at the top of the file; check Prisma's documentation for more info on how to do that. You should also delete the shadowDatabaseURL and previewFeatures lines, unless you're using an old version of Prisma, which you shouldn't be doing :)

We'll also add a field for the Stripe customer ID. This will give us a method to link newly created subscriptions with existing customers in our database. And lastly, we'll add a boolean field isActive to determine if a user should have access to our service. Add these lines inside the User model in the schema:



model User {
   ...
   stripeCustomerId        String?
   isActive                Boolean            @default(false)
}

Lastly, depending on which Authentication provider you want to use, you may need to add some additional fields. Authentication provider refers to services we can use for our users to sign-in with, such as "Sign in with Google" or "Sign in with Facebook." Next-Auth has a long list of built-in providers. For this tutorial we'll use GitHub.

The GitHub provider requires one additional field, so add this to the Account model:



model Account {
   ...
   refresh_token_expires_in       Int?
}

Set Up your Environment Variables

Now that the schema is complete, we need to actually link Prisma to our database. First, add a line which says .env to your .gitignore file. This is EXTREMELY important to make sure you don't actually commit your environment variables and accidentally push them to GitHub later.

Next, create a file called .env in your project folder (not in the Prisma folder). The content to add will depend on your database. For a local Postgres database, you should write the following in your .env.local: postgresql://USER:PASSWORD@HOST:PORT/DATABASE?schema=SCHEMA. To create a new database in psql, run create database subscriptionstutorial or swap out "subscriptionstutorial" for another name specific to your project.

Create the database and Prisma client!

Run npx prisma migrate dev --name init to set up your database. If you hit any syntax issues with the schema, re-check the schema on the Prisma docs and the fields above. If you hit issues with the database connection, check your database through your CLI (for example, using psql for Postgres) to make sure its online and you have the right Database URL.

What just happened?!

Prisma checked your .env for the database URL.
Prisma created and ran SQL commands for you, automatically, to create database tables with columns in a structure that match your schema.
Prisma created the Prisma Client, which contains fully-typed methods for interacting with your database, with the types corresponding to your schema.

Create a dev-safe Prisma Client instance

If we want to actually use Prisma Client to interact with the database, we need to create a client with new PrismaClient(). However, in development mode, hot-reloading can cause the Prisma Client to regenerate too many times.

To fix that, we can use a shared, global Prisma Client in development. Create a file in the prisma folder called shared-client.ts and add this content:



import { PrismaClient } from '@prisma/client';
import type { PrismaClient as PrismaClientType } from '@prisma/client';
let prisma: PrismaClientType;

if (process.env.NODE_ENV === 'production') {
  prisma = new PrismaClient();
} else {
  if (!global.prisma) {
    global.prisma = new PrismaClient();
  }

  prisma = global.prisma;
}

export { prisma }

Set Up Next-Auth

Next, we'll add user account creation to our site. Since we're using Prisma for connecting Next-Auth to the database, and GitHub as our OAuth provider, we'll base the configuration off of the docs pages for the Prisma adapter and the GitHub provider.

First, do npm install next-auth @prisma/client @next-auth/prisma-adapter. The GitHub provider is built-in to next-auth, it doesn't require a separate package.

Delete the file /pages/api/hello.js and add a new file pages/api/auth/[...nextauth].ts.

In the file, add this content:



import NextAuth from "next-auth";
import { PrismaAdapter } from "@next-auth/prisma-adapter";
import GithubProvider from "next-auth/providers/github";
import { prisma } from "../../../prisma/shared-client";

export default NextAuth({
  providers: [
    GithubProvider({
      clientId: process.env.GITHUB_CLIENT_ID,
      clientSecret: process.env.GITHUB_CLIENT_SECRET,
    }),
  ],
  adapter: PrismaAdapter(prisma),
}

To create the GitHub Client ID and Client Secret, go to https://github.com/settings/profile, Developer Settings on the left-hand nav-bar, OAuth Apps, New OAuth App. Fill in a name and your localhost with port for the homepage URL. Copy the homepage URL and add /api/auth/callback/github. This will allow the /api/auth/[...nextauth].ts file to catch this callback URL and use it to create a user in the database. The form should look something like this:

After you create the OAuth app, add the Client ID, a Client Secret, and your local URL into your .env like this:



GITHUB_CLIENT_ID="fill-in-value-from-github-xyz123"
GITHUB_CLIENT_SECRET="fill-in-value-from-github-abc123"
NEXTAUTH_URL="http://localhost:3000"

As an additional convenience, we'll extend the session object to contain the user ID. Add a callbacks field with a session callback function which returns an extended session like this:



export default NextAuth({
  providers: ...,
  adapter: ...,
  callbacks: {
    async session({ session, user }) {
      session.user.id = user.id;
      return session;
    },
  },
}

TypeScript users will also need to extend the session.user type to add this field to it. In the project root, create a file called types.d.ts and add this content there:



import type { DefaultUser } from 'next-auth';

declare module 'next-auth' {
  interface Session {
    user?: DefaultUser & {
      id: string;
    };
  }
}

This is the basic Next-Auth setup - technically, we could now add the frontend sign-up form. But instead, before we get there, we should plan ahead for how we'll connect the user accounts with Stripe.

When we create a user, we'll also create a Stripe customer. This will allow us to easily link customers in our DB to the subscriptions and their payments when customers pay after creating an account.

Set up Stripe

Set up a Stripe Account

On Stripe's website, create a new account and a business. You don't need to enter all of your business information, especially if you don't have it yet! Just enter the minimum info to get started.

Add Stripe to the Project

The part of this tutorial that I spent the most time figuring out was how to connect Stripe customers to our site's accounts. This setup will allow for that.

Add Stripe's node.js SDK to the project with npm install stripe.

Go to https://dashboard.stripe.com/test/apikeys, which should look like this:

On the "Secret Key" row, hit Reveal test key and copy that key into your .env like this:



STRIPE_SECRET_KEY="sk_test_abc123"

You don't need the Publishable key at the moment!

Create a Stripe Customer for Newly Registered Accounts

To accomplish this, we'll use the Next-Auth event system. Events allow Next-Auth to do some custom action after certain user actions like creating a new account or signing in, without blocking the auth flow. Read more about the event system here.

In the [...nextauth].ts file, add the events field as an object with a createUser function like this:



export default NextAuth({
  providers: ...
  adapter: ...,
  callbacks: ...,
  events: {
    createUser: async ({ user }) => {

    });
  }
})

Next-Auth will call this function after a new user account is registered.

Inside of the function, we'll use the Stripe SDK to create a customer, and then add the Stripe customer ID to our saved record for the customer account:



createUser: async ({ user }) => {
      // Create stripe API client using the secret key env variable
      const stripe = new Stripe(process.env.STRIPE_SECRET_KEY as string, {
        apiVersion: "2020-08-27",
      });

      // Create a stripe customer for the user with their email address
      await stripe.customers
        .create({
          email: user.email!,
        })
        .then(async (customer) => {
          // Use the Prisma Client to update the user in the database with their new Stripe customer ID
          return prisma.user.update({
            where: { id: user.id },
            data: {
              stripeCustomerId: customer.id,
            },
          });
        });
    },

Woohoo! If you're with me so far, we've finished the hardest part!

Front-end and Payment form

We're finally ready to build the frontend!

Sign Up Form

Rename pages/index.js to pages/index.tsx and then open that file.

Import the frontend parts of next-auth by adding this line to the top of the file:



import { signIn, signOut, useSession } from 'next-auth/react'

Next-Auth automatically manages and updates the state of the data returned by useSession, so we can use that hook to track the customer's sign-in status and account.

In the exported Home page function, add:



const {data, status} = useSession()

In the tag, which should be empty, add the following content to decide what to render based on the user's status:



<main>
    {status === 'loading' && <p>Loading...</p>}
    {status === 'unauthenticated' && <button onClick={() => signIn()}>Sign In</button>}
    {status === 'authenticated' && <button onClick={() => signOut()}>Sign Out</button>}
    {data && <p>{JSON.stringify(data)}</p>}
</main>

Note: the signIn() function handles both registering for a new account and signing in to an existing account.

We also need to add a global data provider for the useSession hook to connect to. Set this up in _app.js like this:



import "../styles/globals.css";
import { SessionProvider } from "next-auth/react";

function MyApp({ Component, pageProps }) {
  return (
    <SessionProvider session={pageProps.session}>
      <Component {...pageProps} />
    </SessionProvider>
  );
}

export default MyApp;

Test Account Creation

Run the site with npm run dev.

You should see a button which says Sign In.

Click there, then Sign in With GitHub, and follow the authorization flow.

If everything worked, you should be returned to your frontend with the button now reading "Sign Out" and your account data below. Also, if you go to your Stripe Dashboard and check the Customers tab, you should see a row with a new customer that has your GitHub account's email!

Use Stripe To Add Payments

The Approach

Most of our Stripe integration will be powered by a Stripe Checkout page, and Webhooks.

A Stripe Checkout page is a single page that Stripe automatically generates for us, with a payment form that full form functionality, accessibility, and more features. It's a great way to quickly add flexible payments to your site. The one challenge is that it's hosted on Stripe's site, not part of our codebase, so we need some way to send data from Stripe back to our system after a customer purchases a subscription on the Stripe Checkout Page.

To solve that problem, we'll use webhooks. A webhook is nothing super new - it's an API endpoint in OUR system that an EXTERNAL system can use to communicate with our system. In our case, the webhook API endpoint will allow Stripe to "hook" into our system by sending some data for our server to process and handle.

In summary: after creating an account, we'll redirect new users to the Stripe Checkout page for them to pay. Then Stripe will call our webhook to send some data back to our system, and we'll update the database based on that data.

Get the Stripe CLI

To watch all of the events that Stripe sends over webhooks in real time, we'll use the Stripe CLI so that Stripe can post its events to our local devices.

Follow the instructions here to install the Stripe CLI.

Next, follow Step 3 here to connect Stripe to your local server. Use the URL http://localhost:YOUR_PORT/api/webhooks/stripe which we will create in the next step. For example, mine is http://localhost:3000/api/webhooks/stripe.

When you get the CLI installed and started, copy the "webhook signing secret" which the CLI will print into a temporary note.

Create the Webhook

Create a new file pages/api/webhooks/stripe.ts.

Since we are using a public-facing webhook, we have a small problem: imagine if a hacker found this Stripe webhook and sent some fake data about a payment - they could trick our system into giving them access to the benefits of a paid subscription.

Therefore, before we can trust data from a Stripe webhook, we need to check if the request actually came from Stripe. After we verify the call is from Stripe, we can read the data and take some action.

This post by Max Karlsson explains the Stripe verification process in Next.js API Routes really well, so I won't go through it in detail. I'll just include my final webhook code here, which verifies the webhook data and then uses Prisma to update the user to isActive=true when they've paid:



import type { NextApiRequest, NextApiResponse } from 'next';
import { buffer } from 'micro';
import Stripe from 'stripe';
import { prisma } from '../../../prisma/shared-client';

const endpointSecret = // YOUR ENDPOINT SECRET copied from the Stripe CLI start-up earlier, should look like 'whsec_xyz123...'

export const config = {
  api: {
    bodyParser: false, // don't parse body of incoming requests because we need it raw to verify signature
  },
};

export default async (req: NextApiRequest, res: NextApiResponse): Promise<void> => {
  try {
    const requestBuffer = await buffer(req);
    const sig = req.headers['stripe-signature'] as string;
    const stripe = new Stripe(process.env.STRIPE_SECRET_KEY as string, {
      apiVersion: '2020-08-27',
    });

    let event;

    try {
      // Use the Stripe SDK and request info to verify this Webhook request actually came from Stripe
      event = stripe.webhooks.constructEvent(
        requestBuffer.toString(), // Stringify the request for the Stripe library
        sig,
        endpointSecret
      );
    } catch (err: any) {
      console.log(`⚠️  Webhook signature verification failed.`, err.message);
      return res.status(400).send(`Webhook signature verification failed.`);
    }

    // Handle the event
    switch (event.type) {
      // Handle successful subscription creation
            case 'customer.subscription.created': {
        const subscription = event.data.object as Stripe.Subscription;
        await prisma.user.update({
          // Find the customer in our database with the Stripe customer ID linked to this purchase
          where: {
            stripeCustomerId: subscription.customer as string
          },
          // Update that customer so their status is now active
          data: {
            isActive: true
          }
        })
        break;
      }
      // ... handle other event types
      default:
        console.log(`Unhandled event type ${event.type}`);
    }

    // Return a 200 response to acknowledge receipt of the event
    res.status(200).json({ received: true });
  } catch (err) {
    // Return a 500 error
    console.log(err);
    res.status(500).end();
  }
};

With me still? Just a few more steps 😃

Create your Subscription Plan in Stripe

In order for our customers to subscribe to a subscription, we need to actually create the payment plan in Stripe. Go to the Products tab in Stripe. Click "Add Product" in the top right and fill out the form with a name and any other info you want to add. For a subscription model, in the Price Information section, make sure to choose "Pricing Model: Standard", select "Recurring", choose your Billing period (how often the customer is charged, renewing the subscription) and enter a price. It should look something like this:

When you're done, press "Save Product". You're taken back to the product tab, where you should click on the row of the product you just added. Scroll to the "Pricing" section on the Product page, and copy the Price's "API ID" into a note file. It should look something like price_a1B23DefGh141.

Add an Endpoint to Create Payment Pages for Users

Stripe will host the payments page, but we want to dynamically generate that page for each user, so that we can automatically link it to their pre-existing Stripe Customer ID, which is linked to their User Account in our database. (phew, that's a mouth-full).

Remember much earlier, when we added the User ID to the Session? That will become useful now so that we can link the Checkout Page to the user in the current session.

Add a file pages/api/stripe/create-checkout-session.ts. Add this content to the file, which includes some error handling:



import type { NextApiRequest, NextApiResponse } from 'next';
import { getSession } from 'next-auth/react';
import Stripe from 'stripe';

export default async (req: NextApiRequest, res: NextApiResponse) => {
  const stripe = new Stripe(process.env.STRIPE_SECRET_KEY as string, {
    apiVersion: '2020-08-27',
  });

  // This object will contain the user's data if the user is signed in
  const session = await getSession({ req });

  // Error handling
  if (!session?.user) {
    return res.status(401).json({
      error: {
        code: 'no-access',
        message: 'You are not signed in.',
      },
    });
  }

  const checkoutSession = await stripe.checkout.sessions.create({
    mode: 'subscription',
 /* This is where the magic happens - this line will automatically link this Checkout page to the existing customer we created when the user signed-up, so that when the webhook is called our database can automatically be updated correctly.*/
    customer: session.user.stripeCustomerId,
    line_items: [
      {
        price: // THE PRICE ID YOU CREATED EARLIER,
        quantity: 1,
      },
    ],
    // {CHECKOUT_SESSION_ID} is a string literal which the Stripe SDK will replace; do not manually change it or replace it with a variable!
    success_url: `http://localhost:3000/?session_id={CHECKOUT_SESSION_ID}`,
    cancel_url: 'http://localhost:3000/?cancelledPayment=true',
    subscription_data: {
      metadata: {
        // This isn't 100% required, but it helps to have so that we can manually check in Stripe for whether a customer has an active subscription later, or if our webhook integration breaks.
        payingUserId: session.user.id,
      },
    },
  });

  if (!checkoutSession.url) {
    return res
      .status(500)
      .json({ cpde: 'stripe-error', error: 'Could not create checkout session' });
  }

  // Return the newly-created checkoutSession URL and let the frontend render it
  return res.status(200).json({ redirectUrl: checkoutSession.url });
};

Why don't we need signature verification here? The data is coming from our frontend, not Stripe. Ok, but do we need to verify the request is actually from our frontend? No, because this endpoint doesn't have any ability to update the customer status in the database. If a 3rd party managed to call this endpoint, all that they would get is a link to a payment page, which doesn't provide them any way around paying for our subscription.

Get a Checkout URL on the Homepage and Send the User There

Back in your frontend code, go back to the homepage in index.tsx. We need to request a checkout URL to redirect users to.

Add this code into your homepage:




  const [isCheckoutLoading, setIsCheckoutLoading] = useState(false);

  const goToCheckout = async () => {
    setIsCheckoutLoading(true);
    const res = await fetch(`/api/stripe/create-checkout-session`, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
      },
    });
    const { redirectUrl } = await res.json();
    if (redirectUrl) {
      window.location.assign(redirectUrl);
    } else {
      setIsCheckoutLoading(false);
      console.log("Error creating checkout session");
    }
  };

Now to actually use it, we're going to rewrite what we show to signed-in users.

Find {data && <p>{JSON.stringify(data)}</p>} in your homepage code, and change it to this:



{data && (
          <div>
            <p>{JSON.stringify(data)}</p>
            <p>Add a payment method to start using this service!</p>
            <button
              onClick={() => {
                if (isCheckoutLoading) return;
                else goToCheckout();
              }}
            >
              {isCheckoutLoading ? "Loading..." : "Add Payment Method"}
            </button>
          </div>
        )}

Test it all out!

To check whether or not it works, we'll need isActive to be included in the session. Follow these steps to implement it:

add isActive: boolean; to the user type in types.d.ts
Update the [...nextauth].ts session callback to match the following: ```ts

callbacks: {
async session({ session, user }) {
session.user.id = user.id;

  const dbUser = await prisma.user.findFirst({
    where: {
      id: user.id,
    }
  })

  session.user.isActive = dbUser.isActive;

  return session;
},


**Steps to Test the Full Integration:**

0. Check your Stripe CLI to ensure it's still running. If it isn't, re-run it and make sure the signing secret is up-to-date in your webhook file.

1. with the site running, go to the frontend. Press Sign In and you should see this page: 

![Plain page with button that reads "sign in with github"](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/itcyhponmnaaqj2iy91i.PNG)

2. Press the button and you should be taken to GitHub, where you should grant access to the OAuth app.

3. You should then be redirected to the homepage, where you'll see `isActive: false` in the user data, because we didn't add a payment method yet.

4. Press "Add Payment Method" and you should be taken to the Stripe Checkout page!

5. Confirm that the rate and billing interval is correct on the left side of the page. On the right side, enter `4242424242424242` as the credit card number, one of [Stripe's test numbers](https://stripe.com/docs/testing). Enter any Expiration month as long as it's in the future. Enter any CVC, Zip, and name, and press Subscribe.

6. After a brief loading period, you should be directed back to your homepage, with one major change: `isActive` is now true! 🎉🎊

## Debugging

If it didn't work, try these debugging tips:

1. Make sure all of your environment variables are correct.
2. In the `callback.session` function, `console.log` the user argument, the DB user found via Prisma, and the created-Stripe user. Check if any of them are missing fields.
3. Add `console.log` logging in the webhook and in the create-checkout-session endpoint until you figure out what the issue is.
4. If you need to re-test the flow, you will probably need to clear your database. You can do that with Prisma using `npx prisma migrate reset`.

## Conclusion + Next Steps

Congratulations! I hope you were able to successfully implement this complex integration. You now have a system for registering users and collecting recurring payments from them. That's basically a super-power in the web world 🦸‍♀️🦸‍♂️

There are a few more steps you would need to take before you can "go live" with this system:

1. You need to handle the Stripe events for users cancelling their subscriptions or failing to pay (credit card declined, for example). You can handle those cases in the `webhooks/stripe.ts` file, by adding more cases where we currently have the comment `// ... handle other event types`. Here, you should also handle the case when a payment fails after a subscription is created. See [this Stripe doc page](https://stripe.com/docs/billing/subscriptions/webhooks) for more details.

2. You need to host your site, so that you can connect Stripe to the hosted webhook instead of the localhost forwarded-webhook. You can add the deployed webhook URL here: [https://dashboard.stripe.com/test/webhooks](https://dashboard.stripe.com/test/webhooks).

3. For the redirection URLs to support both development and production, in the create-checkout-session endpoint, you can use a condition like `const isProd = process.env.NODE_ENV === 'production'` and then use the `isProd` variable to choose the redirection URL - either localhost or your deployed site.

4. Style the sign-in page. Right now it's pretty dark and bland :)

There are lots more customizations you can make here of course, such as including extra metadata in the Stripe objects, or connecting the payment plans to organizations instead of accounts, or adding multiple tiers of pricing and database fields to track that. 

No matter where you go from here, you should now have a basic framework for the authentication and payment parts of your subscription services!

## Connect with Me

Thanks for reading! I hope this saved you some time and frustration from the process I went through to get this all set up.

Feel free to leave a comment if you have a question, or message me on [Twitter](https://twitter.com/ajones55555). I would also really appreciate if you could check out the project I'm working on which inspired this article, [Envious](https://envious.dev/) 🤩

Let me know what tutorial you'd like to see next!

Adding Personality to a Boring Form in my Saas Project

Andrew Jones — Sat, 08 Jan 2022 21:30:13 +0000

In this series, I'm building a Software-as-a-Service product for syncing environment variables between team members. Since my background is mostly frontend, I'll be learning about backend technologies on the way, along with some UI/UX. You can read more about my concept and technology selection in my first post here.

The Challenge

Users of this product will be able to create multiple "Projects," which each have "environments" for managing variables. For example, for a Node.js API project, you might have a project called "My API Server" with two environments, "develop" and "production."

I started building the form for creating projects, using react-hook-form and some simple inputs. It came out like this:

...unfortunately, this was the most boring form I had ever seen! I decided I needed to redesign this to make it feel more like a polished, professional project than a bunch of text fields thrown together.

The Thought Process

I thought about varying it up a little bit by changing the Name field width and putting it on the same line as the Name label. While that helped, it was still super boring.

Then I took a step back. I started thinking about the big picture product. So I started thinking, how will the data from this form actually be displayed?

I threw this together quickly, showing projects as tiles with the project name and environments (plus I could've added the description).

Don't get me wrong, this design is ALSO super boring, but it gave me an idea: what if I added an icon to the project? For people who have a lot of projects, this would help them make a visual distinction between projects, without affecting the core functional goals of the project. Plus, it would make the UI look more user-friendly than a plain text dev console.

A Step in the Right Direction

I didn't want to create an image upload system for this simple feature: that would be too much work for me (as the developer) and for the users creating a project. But then what images could I use? Maybe an icon library? Or...

Emoji! Emoji are fun, available on all modern devices, and there's enough variety for people to choose an emoji related to their project. Maybe later I would also add the ability for users to pick a Letter/Color icon instead.

I explored some React emoji pickers online so I wouldn't have to build it myself, and eventually found Emoji Mart.

After adding the emoji picker and a display field, the Project Creation form looked like this:

That's... better... but still mostly boring, and the emoji looks weird as an afterthought.

Again, I thought about the big picture: how would I display this emoji? I imagine the layout of the project list display like this: the selected emoji, large enough to see on a quick glance, next to the name and description of the project.

Then I thought, why not display the form like that too? Then the project creation form would have conceptual symmetry with the project display list.

The Solution

I rewrote the project creation form to look more like a single row, with the currently-selected emoji next to the name and display fields, in the same place as they would be displayed. This was the final product:

I also made the Call-to-Action more distinct by using a bright background, and I added a scale-up micro-interaction on-hover.

You may have noticed the Project Key field missing. At first I tried adding it below the row, because the project key won't be displayed in the row later, but it looked weird being the only element outside. I also wanted to add some more information around the project key, so I moved the field to a modal which pops-up when "Create" is pressed. I'll write more about that in another post!

Conclusion

I'm really happy with the end result! It made the form more interesting by adding personality and making it sleek, and made the product overall feel more polished.

Thanks for reading! I hope this gave you some ideas for your own projects!

Follow me here or on Twitter for more updates and other content. Feel free to DM me for questions!

How I made an iOS 15 Safari Extension to get rid of Annoying Reddit "Open in App" Pop-ups

Andrew Jones — Fri, 24 Sep 2021 15:15:41 +0000

iOS 15 revamps the extension system built into iOS and iPad OS. This update lets us build full-fledged extensions for iOS users, including JS/CSS injection, full user-interfaces written in web languages, and more.

In this article, I'll take you through the process of building one of these extensions! If you're just interested in checking out the extension, here is the code, and here it is on the App Store.

The Problem

One annoyance I hit a lot when I'm browsing on my iPhone is the Reddit "Open in App" prompts. Sometimes you can "Continue in Safari" but other times, the prompts are completely blocking, like this:

If I found a Reddit thread or subreddit through a Google search, I probably don't want to open the app. Some people don't like using the app at all. It's a pretty dark design pattern designed to move people to the app, for seemingly no benefit to the user. So we'll create an extension to block these intrusive popups!

Set Up

You'll need macOS Monterey, which is available in a public beta, and the newest version of Xcode, which you can download from the Mac App Store.

Open Xcode and hit New Project. In the template selection menu, search for "safari" and choose Safari Extension App.

Choose a name for your extension, I made mine Viewwit, and hit Next. Choose a folder and hit Create.

The Project Structure

You'll notice two main folders in your project, which represent two targets - the main app, and the Safari Extension.

This project template is designed for web developers! So, the main app has all of the Swift needed for a simple extension already written, and it has a Resources folder with an HTML file, Main.html, for designing the app interface.

For now, we'll focus on the Extension folder. This folder also has a Resources folder with all of the HTML/CSS/JS for a basic extension. It also includes a manifest.json which tells Safari about your extension - which files are used for what, which permissions your extension needs, etc. All of this follows the same standards as Chrome extension development, which is pretty nice because it means there's lots of information online about the manifest fields and extension development.

Functionality

For my extension, I needed to add a script to modify the page HTML. This is a simple extension which doesn't need any background processes, so I commented out the JS in background.js and removed its declaration from the manifest file.

When you open the content.js file, you'll see a message system set up. I also didn't need that, so I deleted it. I replaced this code with simple JS to use document.querySelector to find the elements we need to delete and delete them, and run that function on an interval. For most cases, you would want to use window.onload or the DOMContentLoaded event, but the Reddit pop-ups load after those events, so I needed the interval.

How did I find which elements need to be hidden? If you don't have the Develop menu on your Mac Safari top action bar, go to Safari > Preferences > Advanced > Show Develop menu in menu bar and enable it. Then connect your iOS Device to your Mac (this also works with any open Simulator). Go to Safari on your iOS device and go to any site. Then in the macOS Develop menu, hover over your device, and you can see your iOS device open tabs. Click one to open the full dev tools for your iOS tab on your computer! You can hover over elements, and edit or delete them as needed.

Manifest and Permissions

In manifest.json, fill in the name and description for your extension.

Check the content_scripts field. You should see a URL matcher checking for example.com. Replace that with the domain relevant to your extension, for example:

    "content_scripts": [{
        "js": [ "content.js" ],
        "matches": [ "*://*.reddit.com/*" ],
        "run_at": "document_start"
    }],

(I also added the run_at field to start my interval as early as possible)

I also added "*://reddit.com/*" to the permissions field array.

If you want to match all URLs, add "<all_urls>" to your permissions array

You can test your extension by running it on an iOS simulator or on your device. You need to enable the extension in Safari or in Settings after installing the app.

Preparing the App for Submission

I designed a basic logo in Excalidraw.

Then I used PWA Builder's Image Generator to generate extension Icons from the logo. I renamed the appropriate sizes to match the existing images in Resources/images, and dropped them in that folder to replace the template images.

Finally, I used AppIcon.co, dropped in my 512x512 image from PWA Builder, and generated the icons for my app.

Additionally, I updated the content in both HTML files, the extension popup.html and the app's Main.html. In popup.html I added static content saying the extension is active, and in Main.html I added instructions on how to enable the extension. You can build much more complex functionality in popup.html and connect it to the webpage or your app, but I didn't need that for my use-case. Check out Apple's docs and examples on this for more info!

Lastly, I archived a build of my app, and prepared it for submission in App Store Connect. There are lots of tutorials on how to do this online :)

Conclusion

In this post, we walked through the process of building an iOS 15 Safari extension. We covered how to start a Safari extension, how to work with iOS web dev tools on a Mac, and the basic files involved in building the extension's functionality. This is a really powerful system and I can't wait to see what people build with it!

You can check out the code here, or check out the extension on the App Store here!

Follow me here or on Twitter for more articles and some tweets, sometimes they're even funny! Feel free to DM me for any questions!

Building the Basic UI of my SaaS Product - Update 4

Andrew Jones — Sat, 11 Sep 2021 18:03:46 +0000

Back after a long hiatus! In this series, I'm building a Software-as-a-Service product for syncing environment variables between team members, and with CI/CD pipelines. Since my background is mostly frontend, I'll be learning about backend technologies on the way. You can read more about my concept and technology selection in the first post, project setup in the second post, and adding authentication in the third post. I also wrote this tutorial based on some of what I learned about database setup.

In this post, I'll build out the UI and functionality for viewing the organizations and projects associated with a user, the UI that leads to the core functionality of saving variables.

Approach

Generally as a rule-of-thumb, people say not to spend too much time on the frontend until the core backend functionality is worked out. Since I'm a frontend dev and I didn't want to burn out working through backend work, I decided to work on them both. Sure, I might need to change some of the frontend work if the backend functionality needs changes, but I'll cross that bridge if it happens. More important to keep myself engaged. This approach also helps me figure out what backend APIs I need to implement!

Layout UI

I looked at other dev tool sites for inspiration and landed on a tab menu similar to Vercel's dashboard. I created a Layout component and wrapped all pages with it in my pages/_app.tsx file.

When a user is signed in, the Layout component will show their profile picture in round format. The GitHub auth provider provides the URL for this picture when a user registers.

{session && (
    <div className="float-right h-10 rounded-full flex">
      <span className="flex-1 pt-2.5 mr-5">{session.user?.name}</span>
      <img
        src={session.user!.image!}
        alt="Your Profile Picture"
        className="h-12 rounded-full border-purple-100 border-solid border-2"
      />
    </div>
)}

I also created a NavLink tab component and used it to create the tab menu based on the tabs I imagine the site will need. And finally, I picked a nice dark purple for the background. I may add a light-mode later with the purple used for the text instead of the background.

User Creation Functionality

In the last post I added sign-in and registration through GitHub. I was able to get an account set up and saved in my DB, but there was a problem: this account had no organizations or projects. While I might change this later based on the pricing model, I decided to add a personal organization and starter project to new users (similar to the GitHub personal organization).

Next-Auth supports a custom callback when a user is created, so in my Next-Auth config in pages/auth/[...nextauth].ts file, I added this event:

events: {
    createUser: async (user) => {
      await prisma.organization.create({ //create a new organization in the DB
        data: {
          name: 'My Organization',
          adminUsers: { // create a new adminUser-organization relation using this org and the given userId
            create: {
              userId: String(user.id),
            },
          },
          projects: {
            create: { // Create a new project so the user has something to start with, and link it to this org
              name: 'My First Project',
              ...
            }
          },
          ...
        }
      })
    }
}

Using psql to interact with my database, I deleted my user, and then re-created it using the frontend to test this code. I verified that an organization and project were created and linked to the new user.

Organization Pages and API Routes

Next, I needed a way to fetch the signed-in user's organizations so that the frontend could show a list of their organizations. I added this simple API route which gets the session using Next-Auth, and then searches the database for every organization where the user associated with this session is an admin:

export default async (req: NextApiRequest, res: NextApiResponse) => {
  const userSession = await getSession({ req });
  const orgs = await prisma.organization.findMany({
    where: {
      adminUsers: {
        every: {
          userId: userSession!.user!.id
        }
      }
    },
    include: {
      projects: true
    }
  })
  res.status(200).json({ myOrganizations: orgs })
}

Since I have cookies saved on localhost:3000 in my browser for the sign-in, I can test this API route by navigating to it directly in my browser:

I also tested from an incognito window where I'm not signed in, and as expected, got an internal server error. Why? Because I used this TypeScript non-null assertion (!) here userId: userSession!.user!.id which is really a lie to the compiler - userSession CAN be null, when someone tries to access this endpoint without signing in. So instead of !, we need to add a check for the userSession to not be null and return an error if it is. I added this check above the findMany call:

  if(!userSession?.user){
    res.status(401).json({
      error: {
        code: 'no-access',
        message: 'You need to sign-in to view your organizations.',
      },
    })
    return
  }

Finally, I added a frontend page pages/organizations/index.ts which very simply fetches data from /api/organizations/my-organizations and then displays loading, the data, or any errors based on the page state:

import { useEffect, useState } from 'react';
import { OrgCard } from '@components/OrgCard'
import { OrganizationWithProjects } from '@project-types/organization';

export default function ListMyOrganizations() {
  const [myOrgs, setMyOrgs] = useState<OrganizationWithProjects[]>();
  const [error, setError] = useState();

  useEffect(() => {
    fetch('/api/organizations/my-organizations')
      .then((res) => res.json())
      .then(orgsData => setMyOrgs(orgsData.myOrganizations))
      .catch(setError);
  }, []);

  if(!myOrgs && !error) return <div>Loading...</div>
  else if(error) return <div>{JSON.stringify(error)}</div>
  else return (
    <div>
      {myOrgs!.map((org: OrganizationWithProjects) => <OrgCard organization={org} key={org.id}/>)}
    </div>
  )
}

The result looks like this:

The Organization Page / Project List

I won't get into too much detail here, but I followed roughly the same process to design and implement the Organization "Profile" page. Keeping it simple for now, there's a list of projects and their environments, a free-trial marker, and a list of admins with an "Add" button (not functional yet). I'll probably add a few more "design" elements like borders and colors later.

Conclusion

In this post, we added the basic UI which takes users to the Project page. In the next article, we'll finally implement the core functionality of the site: the project page and secure variable storage.

Follow me here or on Twitter for more updates and other content. Feel free to DM me for questions!

SaaS Project Update 3 - Adding Authentication to my Next.js/Prisma/Postgres Project

Andrew Jones — Wed, 09 Jun 2021 00:26:50 +0000

In this series, I'm building a Software-as-a-Service product for syncing environment variables between team members. Since my background is mostly frontend, I'll be learning about backend technologies on the way. You can read more about my concept and technology selection in the first post, and you can read about my project setup in the second post.

Adding Authentication

The project needs authentication so that users can securely access their organizations, projects, and environments. I'm planning on adding an extra level of encryption for the environment variables themselves, because they need to be extra secure to protect sensitive + powerful API keys and other environment variables. We'll get to that in a later post.

⚠ Authentication is a super tricky feature to get right - there are layers upon layers of complexity and security to consider, too much for most projects to handle well. Generally it's a best practice not to build your own authentication system - there's lots of info online on why that is, but see this article for some examples.

Next-Auth

I started researching authentication solutions. Auth0 is one of the most popular systems in this area - https://auth0.com/blog/create-a-simple-and-secure-node-express-app/

However, reading through the tutorial, I decided I needed something more packaged, so that I would handle even less of the auth and security in my code.

Soon, researching by ~~a serious scientific process~~ googling topics like "Nextjs auth system," I found Next-Auth. Next-Auth is an open-source authentication system for NextJS apps. It's super flexible, and compatible with many authentication providers (which I'll explain in a minute) and multiple database systems, including postgres.

Next-Auth Setup

We need to connect an "Authentication Provider" to Next-Auth. The best way to explain "authentication providers" is through an example: when you go to a new site and see "Sign in With Google," Google is acting as an authentication provider for that site. You essentially create an account for the new site by signing-in through Google, which provides credentials to the new site.

I decided to use GitHub as my first authentication provider, which will work well because this is a tool made for developers. With Next-Auth, I can always add a new provider later. I followed the Next-Auth guides for the GitHub provider and the Prisma adapter.

First, npm i next-auth @next-auth/prisma-adapter@canary.

(Note: If you're following along, don't miss the @canary or you'll have an outdated package with different usage than the current version!)

Then I created this file: src/pages/api/auth/[...nextauth].js with this content:

import NextAuth from "next-auth"
import Providers from "next-auth/providers"
import { PrismaAdapter } from "@next-auth/prisma-adapter"
import { PrismaClient } from "@prisma/client"

const prisma = new PrismaClient()

export default NextAuth({
  providers: [
    Providers.GitHub({
      clientId: process.env.GITHUB_CLIENT_ID,
      clientSecret: process.env.GITHUB_CLIENT_SECRET
    })
  ],
  adapter: PrismaAdapter(prisma),
})

To get the GITHUB_CLIENT_ID and GITHUB_CLIENT_SECRET, you can go to GitHub, click your profile picture in the top right corner, click Settings in the dropdown menu, click Developer Settings on the left-hand menu, and then go to OAuth Apps and create a new OAuth App. Enter your project information, and use this URL as the Authorization callback URL, changing the port to your port: http://localhost:8080/api/auth/callback/github

Linking to the Database

We set up the Prisma connector so that Next-Auth would use Prisma functions to manage users in the database, so now we need to add required fields to the Prisma schema to create the tables in the database and the required Prisma functions. I added the models here to my Prisma schema: https://next-auth.js.org/adapters/prisma#setup. I also added the fields my project needed to the User model, like projects to connect the User to the projects they have access to.

Then I ran npx prisma migrate dev --name "Added users to schema" to update my database and generate the required prisma functions.

Adding UI to Test the Functionality

Finally, we can add the basic Next Auth UI to actually test the authentication flow. I ran npm run dev to start the site and deleted all of the content on the homepage (src/pages/index.tsx). I copied the pages/index.ts file from the Next Auth "Example Code" section to get a basic auth flow setup with sign in and sign out buttons, and text to tell us our auth status.

I launched the site, and when I click sign-in, I get this view:

...but the button didn't work! 🤯 I got a blue error message from Next Auth.

Troubleshooting

In my server terminal, I got this error:

https://next-auth.js.org/errors#oauth_callback_handler_error Error:
Invalid `prisma.account.findUnique()` invocation:

{
  where: {
    providerId_providerAccountId: {
      providerId: 'github',
      providerAccountId: 18769232
                         ~~~~~~~~
    }
  },
  select: {
    user: true
}

Argument providerAccountId: Got invalid value 18769232 on prisma.findUniqueAccount. Provided Int, expected String.

To fix this, in my schema I changed the type of Account.providerAccountId from String to Int, and a migration fixed this bug.

After relaunching, when I click the button.... it doesn't work again! I got redirected from localhost:3000/... to localhost:8080/... so I fixed the problem by adding NEXTAUTH_URL="http://localhost:8080/" to my .env.

Finally, when I hit "Sign in with Github," I get to this:

And after allowing access, I was redirected to my homepage, now signed in with the email linked to my GitHub account:

Conclusion

In this update, I added authentication to my NextJS/Prisma/Postgres app with Next-Auth.

In the next update, I will create some API routes to implement the project's business logic, and get started on the UI.

Follow me here or on Twitter for more updates and other content. Feel free to DM me for questions!

Backend Basics for Frontend Engineers: Dive into SQL and APIs with NodeJS, Prisma, and PostgreSQL

Andrew Jones — Mon, 31 May 2021 20:10:14 +0000

This article is written for someone with 0 backend experience, so anyone should be able to learn from it!

Working on a full-stack project recently, I found some great tools that allow devs to easily generate database structures, handling much of the complexity for you. In this article, we will create a simple express API using NodeJS, Prisma, and PostgreSQL (don't get scared by the list of technologies!)

SQL vs NoSQL

SQL and NoSQL are two types of databases.

SQL

SQL uses data tables, sort of like an Excel or Google Sheets spreadsheet. In relational databases, tables can be linked to each other through relation fields.

For example, let's consider a database storing info about cars and their owners. It could be structured like this:

`owner` table

id	name	age	carId
1	Steven	24	143
2	Bob	41	144
3	Sarah	35	145

`car` table

id	make	model	year	ownerId
143	Chevrolet	Corvette	2020	1
144	Honda	Civic	2010	2
145	Hyundai	Palisade	2017	3

In this example, the carId column in the owner table refers to the id column of the car table. The ownerId column of the car table refers to the id column of the owner table. id is the primary key in each table, and ownerId and carId are "foreign keys" because they refer to a different table's primary key.

NoSQL

NoSQL is a document-based type of database, where generally documents are similar to JSON objects. This structure might feel more familiar to frontend developers, but both types of databases have advantages and disadvantages. We'll use SQL in this tutorial, so I won't go into NoSQL too much here. You can read more about their differences online by searching for comparisons!

PostgreSQL and Database Installation

PostgreSQL (sometimes called Postgres) is an extremely popular relational database following SQL specs. It allows us to set up databases with tables and connect them with easily understandable relations.

First, we'll install Postgres so that you can set up databases on your computer. You can find the download for your OS here: https://www.postgresql.org/download/.

When you go through the download, stick with default settings. You will be asked for a password for the root user: no matter what you choose, ⚠make sure to write down the password you selected⚠. The default username is postgres.

After the installation, you can log in to your local database instance and play with it on the command line! Search your computer for a program called psql, which is a Terminal/CMD prompt application that lets you interact with your postgres database. Open the program and login by pressing Enter to use the defaults and then entering your password. You should see this when you're done:

Now we can test out some commands to get familiar with Postgres. First, type \l and then Enter to list the existing databases on your machine. Postgres should come with a few created by default.

While I said we wouldn't write a single SQL query, I would recommend creating a separate database for this project. In PSQL, type CREATE DATABASE prismademo; and press Enter. Then you can connect to the new database with \c prismademo.

You can spend some time creating databases and tables if you want to - there are plenty of simple guides you can find by searching "how to create databases in psql", "how to create tables in psql", etc.

⚠Heads up! Most SQL queries you enter in PSQL need to end in a semicolon. Don't forget that part!⚠

Server Setup

In a normal terminal or CMD prompt (not psql), create a folder for your JavaScript code that we will use to create the API server. I named my folder prisma-demo. cd into that folder, and run npm init. Follow through the steps of initializing your project. Open your project folder in your favorite IDE (if you use VS Code, you can type the command code . in this directory to open VS Code here). You should see a package.json with the info you just entered.

Prisma Integration

Install Prisma

We're finally at the keystone, Prisma, which will:

connect our database and JS server
generate tables in our database for us
generate JavaScript functions and TypeScript types for interacting with our database (no worries if you don't use TypeScript, keep reading!)

First, do npm install prisma -D to install Prisma.

Next, do npx prisma to verify Prisma is installed, and then npx prisma init. The second command will generate a Prisma schema file in prisma/schema.prisma and a .env with a sample database URL.

⚠ Next we should add the .env to .gitignore. Create a .gitignore file with this content:

.env
node_modules

Now you can safely run git init and commit your project if you want to.

Fix the database URL

Open your .env and replace the sample URL with this one:

DATABASE_URL="postgresql://postgres:INSERTYOURPASSWORD@localhost:5432/prismademo?schema=public"

If you didn't create a prismademo database earlier, replace that part with postgres before ? in the url.

Fill in the Prisma Schema

Open schema.prisma. Here, we can define data models, and then use the Prisma CLI to generate the required database tables and JavaScript functions based on the data models.

We'll use the car example from above. We'll say that each person can own multiple cars, and each car can only have one owner, meaning owners/cars have a one-to-many relationship. You can read about all of the possible types of relationships here: https://www.prisma.io/docs/concepts/components/prisma-schema/relations

Add these lines to your schema to set up the models. Note the relation: each owner has a list of cars, and each car can have one owner, specified by ownerId. Note that it looks sort of similar to a GraphQL Schema!

model Owner {
  // Prisma will automatically create an ID for each entry for us
  id   Int    @id @default(autoincrement())
  name String
  age  Int
  // Each Owner can have multiple cars
  Car  Car[]
}

model Car {
  id      Int    @id @default(autoincrement())
  make    String
  model   String
  year    Int
  // Each Car can have only 1 owner
  ownerId Int
  owner   Owner  @relation(fields: [ownerId], references: [id])
}

We use the @id annotation to signify that this is the primary key field for this model. @default(autoincrement()) means that we don't need to provide an ID when we create an owner or car, it will be automatically assigned to 1 + the last ID.

Heads up! There's a great VS Code extension to help format and maintain this file: https://marketplace.visualstudio.com/items?itemName=Prisma.prisma

Migrate the Schema to the Database

Now that we've created the Prisma Schema, we need to run a command so that Prisma will actually create the tables for this schema in the database. Run npx prisma migrate dev --name "Create initial schema" in your project directory terminal/cmd prompt. You can change the name text, but keep it descriptive! (Note: if the command freezes at "Running generate..., stop the process and run npx prisma generate.)

Go back to your db in PSQL. Now run \dt, and you should see the tables created from our Prisma Schema! How cool is that? 😎

You can also run npx prisma studio to view your database structure in an interactive web view. I would recommend trying this so that you can see the generated database visually!

When you run a migration or prisma generate, Prisma also creates fully-typed JavaScript functions for us to interact with the database without writing any SQL manually! Let's see how we can use those functions to create an API.

REST API to Create an Owner and a Car

If you're a JS developer, you may already be familiar with this part. Express is an NPM package for writing servers.

Run npm i express body-parser to install express and body-parser, which we'll use to read JSON input in the API.

Create a server.js file in your project root folder. Add this content:

const express = require("express");
const app = express();
const bodyParser = require('body-parser')
const port = 3030;

const jsonParser = bodyParser.json()

app.listen(port, () => {
  console.log(`Example app listening at http://localhost:${port}`)
})

First, we'll create a REST API route to create a user and a car at the same time and link them together.

Add the Prisma Client import and instantiation to your server.js file, outside of any route, underneath const port = ...

const { PrismaClient } = require('@prisma/client')
const prisma = new PrismaClient()

Next, create a POST route:

app.post('/owner', jsonParser, async (req, res) => {

})

When a web-app or REST client sends a POST request to this express server with the URL we specified, the server will run this function.

We'll use the functions which Prisma generated to create an owner and a car in our database.

The Prisma create functions follow the form prisma.<MODEL NAME>.create, so in our case we'll use prisma.owner.create(...). If you're following along, you'll see that our IDE knows that our model is called owner and autosuggests it, because Prisma has generated these functions from our schema.

Prisma will suggest fields for the owner model in the data argument field, like this:

app.post("/owner", jsonParser, async (req, res) => {
  const body = req.body
  const newOwner = await prisma.owner.create({
    data: {
      name: body.name,
      age: body.age
    }
  })
}

Another great feature of Prisma is that we can create the Car database entry and link it to this Owner in the same function call. This is how:

const newOwner = await prisma.owner.create({
      data: {
        name: body.name,
        age: body.age,
        Car: {
          create: {
            make: body.car.make,
            model: body.car.model,
            year: body.car.year,
          },
        },
      },
    });

Lastly, we'll add try/catch for error handling, and a response:

app.post("/owner", jsonParser, async (req, res) => {
  const body = req.body
  try {
    const newOwner = await prisma.owner.create({
      data: {
        name: body.name,
        age: body.age,
        Car: {
          create: {
            make: body.car.make,
            model: body.car.model,
            year: body.car.year,
          },
        },
      },
    });
    res.status(200).json({owner: newOwner})
  } catch (e) {
    console.log(e);
    res.status(503).json({error: "Failed to create Owner and Car."})
  }
});

Something to Notice: We didn't specify ownerId in the car JSON, or carID in the owner JSON. Since we are creating these with one nested in the other in one function call, Prisma will link the IDs for us! This is enabled by the @relation fields we added in the schema.

Testing it Out

Run node server.js in your project directory to run your server. You can then use any HTTP Client to test this API route. For example, you can use cURL, Postman, or a NodeJS file with node-fetch.

No matter how you test it, run a request similar to the following POST request, and you should get the given response!

If you also want to get the car information in the response, you can add the include field in your prisma.owner.create call to tell Prisma to include the connected Car object:

const newOwner = await prisma.owner.create({
      data: {
        ...
      },
      include: {
          Car: true
      }
    });

Restart the Node server after adding this field, and re-run the request to see the created car!

🎉Congratulations! At this point, you've set up a database, created tables from a schema, and added a REST API to store information in the database.

View your Database

Back in the psql terminal, you can run TABLE "Car"; to print out the Car table. You should see output in tabular form like this:

prismademo=# TABLE "Car";
 id |   make    |  model   | year | ownerId
----+-----------+----------+------+---------
  1 | Chevrolet | Corvette | 2020 |       1
  2 | Chevrolet | Camaro   | 2020 |       2
(2 rows)

You can also view the data in Prisma Studio by running npx prisma studio again!

Add a Route to GET Stored Info

Lastly, we'll add a REST API route to GET a car, so your frontend can read car data. The requested Car ID will be part of the URL.

The find___ functions in Prisma allow us to use a filter to get database entries meeting a certain condition. We'll use prisma.car.findFirst(...) to find the first car with the given ID.

Underneath the POST route we added, add:

app.get("/car/:carId", async (req, res) => {
  try {
    const car = await prisma.car.findFirst({
        where: {
            id: Number(req.params.carId)
        },
        include: {
            owner: true
        }
    })
    res.status(200).json({ car });
  } catch (e) {
    console.log(e);
    res.status(503).json({ error: "Failed to get car." });
  }
});

We name the URL parameter carId using the :carId in the URL. We don't need bodyParser because we don't need to parse POSTed JSON, like we did when we created the owner/car.

We use where to tell Prisma to get the car with the matching ID. You'll notice autosuggest is supported here too. We use Number(...) to convert the URL parameter from a string to a number, because the ID is an Int according to our schema. Finally, we use include to get the owner details with the car details.

Restart your server. You should be able to go to http://localhost:3030/car/1 in your web browser or HTTP client to view the first car you created!

Recap and Taking it Further!

That was a lot of information, very fast. Great work so far!

To sum it up, in this tutorial you:

Installed PostgreSQL
Created a database for your project
Created database tables from a Schema using Prisma
Wrote an express server REST API
Added routes to the API to create/store and get database entries.

Some ideas for taking this further:

Try adding new models in the schema (maybe a Dealership, or create a model for the Make instead of just astring)
New types of relations, or new field types. Maybe a purchasedAt date field on the car, or allow a car to have more than one owner with a many-to-many relationship.
Add more API functionality. For example, you can add a route to add another car to an existing owner, or delete a car from an owner.

I would definitely recommend checking out the Prisma documentation, especially different types of relations, data types, tradeoffs of Prisma, and more: https://www.prisma.io/docs/

You can also view my source code here: https://github.com/aej11a/prisma-demo

Thanks for reading! Follow me here or on Twitter for more updates and other content. Feel free to DM me for questions!

SaaS Project Setup & Database Creation - Update 2

Andrew Jones — Sat, 29 May 2021 16:34:18 +0000

In this series, I'm building a Software-as-a-Service product for syncing environment variables between team members. Since my background is mostly frontend, I'll be learning about backend technologies on the way. You can read more about my concept and technology selection in my first post here.

Boilerplate

I knew going into this project that I wanted to install tools to keep my code clean and high quality - prettier, ESLint, and Husky for automatically running those tools.

Instead of spending lots of time configuring everything myself, I found this open source project starter on GitHub: Boilerplate and Starter for Next JS 10+, Tailwind CSS 2.0 and TypeScript.

Personalizing the Template

ESLint Config

As soon as I started writing code with this template, I decided the ESLint config was too strict for my taste. I replaced the ESLint config with a simpler version, based on this article with some modifications based on my experience with ESLint:

module.exports = {
  root: true,
  env: {
    node: true,
    es6: true,
  },
  parserOptions: { ecmaVersion: 8 }, // to enable features such as async/await
  ignorePatterns: ['node_modules', '.next', '.out', '!.prettierrc.js'],
  extends: ['eslint:recommended'],
  overrides: [
    // This configuration will apply only to TypeScript files
    {
      files: ['**/*.ts', '**/*.tsx'],
      parser: '@typescript-eslint/parser',
      settings: { react: { version: 'detect' } },
      env: {
        browser: true,
        node: true,
        es6: true,
      },
      extends: [
        'eslint:recommended',
        'plugin:@typescript-eslint/recommended', // TypeScript rules
        'plugin:react/recommended', // React rules
        'plugin:react-hooks/recommended', // React hooks rules
        'plugin:jsx-a11y/recommended', // Accessibility rules
      ],
      rules: {
        // We will use TypeScript's types for component props instead
        'react/prop-types': 'off',

        // No need to import React when using Next.js
        'react/react-in-jsx-scope': 'off',

        // This rule is not compatible with Next.js's <Link /> components
        'jsx-a11y/anchor-is-valid': 'off',

        '@typescript-eslint/no-unused-vars': ['warn'],

        // I suggest this setting for requiring return types on functions only where useful
        '@typescript-eslint/explicit-function-return-type': [
          'warn',
          {
            allowExpressions: true,
            allowConciseArrowFunctionExpressionsStartingWithVoid: true,
          },
        ],
      },
    },
  ],
}

Next.js Folder Structure

Next.js gives developers the option of keeping their source files in the project root or in a /src directory. Since I'm keeping everything in subfolders anyway, ~~I moved the subfolders from /src to the project root. Personal preference 🤷‍♂️~~
EDIT: I realized that the postcss purge configuration used the src directory so that we don't need to hardcode each folder we want the purger to check. As a result, I reverted this change.

Database Setup

The juicy part!

I went through the Prisma Start From Scratch guide.

Installing PostgreSQL

I installed PostgreSQL from the official website, and started playing with it locally. After installing it, you can launch PSQL, a Terminal/CMD prompt which lets you interact with your local Postgres server.

I used CREATE DATABASE <project name>; to create a new database. Don't forget the ;, I missed that at first and I was so confused! 😅

You can use \l to list all the databases you have created. Then, you can use \connect <project name> or \c <project name> to connect to the database you just created.

I'll do most of the database setup in Prisma, but for now I created a table for testing, again using PSQL:

CREATE TABLE USER(
   ID INT PRIMARY KEY     NOT NULL
);

You can list the tables in your database using \dt.

Prisma Setup

Next, I went back to my project, and followed the rest of the steps from the Prisma Getting Started Guide. I won't go into that in too much detail since it's all included in the Prisma link above :)

I filled in the schema with a rough version of how I imagine my system working. I'm imagining organizations which pay for the service. Each organization will have projects, and each project can have different environments, each with their own set of variables. Users will be linked to projects so that organizations can control who has access to which variables.

This is a snippet from the schema I created based on this, with explainer comments:

model Project {
  // Using @default will allow Prisma to create the IDs for us when a new entry is added to the Projects table
  id              Int               @id @default(autoincrement())
  name            String
  description     String
  environments    Environment[]
// Each project can only have one organization, so the project has an organizationId
  organizationId  Int
// This will allow us to retrieve info from the DB about the organization with organizationId equal to the field above this line
  organization    Organization      @relation(fields: 
[organizationId], references: [id])
  usersWithAccess UsersOnProjects[]
}

// Users can have multiple projects and projects can have multiple users, so we use a many-to-many relationship via this model.
model UsersOnProjects {
  user      User     @relation(fields: [userId], references: [id])
  userId    String
  project   Project  @relation(fields: [projectId], references: [id])
  projectId Int
  createdAt DateTime @default(now())

  @@id([userId, projectId])
}

Heads up! Prisma has a great VSCode extension for formatting and validating the prisma.schema file!

After running npx prisma migrate and npx prisma generate, Prisma has now set up all of the required database tables for my schema, and also created fully-typed functions in the Prisma client specific to my model.

For example, I can write code like this to write to the database:

import { PrismaClient } from '@prisma/client';
const prisma = new PrismaClient();
prisma.project.create({
    data: {
         name: "My Project",
         description: "My New Project!!!",
         organizationId: 2
    }
})

Since Prisma generated the project.create function from my schema, it knows the expected types of the inputs, and my IDE will autosuggest the fields that I just put in my schema! I think that's pretty cool.

Finally, you can run npx prisma studio for a web interface where you can view all of your database tables visually. Here's the project I just created:

Thanks for Reading!

In the next part, I'll add some API Routes so that we can see the magic of Prisma, and then we'll add authentication!

Follow me here or on Twitter for more updates and other content. Feel free to DM me for questions!