DEV Community: Noah Falk

Making a REST API typesafe with React Query and Zod

Noah Falk — Wed, 31 Jul 2024 22:39:47 +0000

I have been a proponent of using tRPC for building end-to-end type-safe APIs for a long time now. It's an amazing library that allows you to directly call your backend functions from the frontend with full type safety. It infers the types automatically, so you won't have to worry about code generation like with GraphQL. In addition, it wraps React Query. This means you'll get to enjoy all the state management benefits that React Query provides.

The big drawback is that tRPC requires not only your frontend but also your backend to be written in TypeScript. And even if that is the case, you'll also need to put everything into a monorepo for type inference between frontend and backend. On many projects, that's a non-starter, since you might already have a fully defined REST API in a completely different technology. So let's assume we're working with a REST API written in Django that we'll call from our React frontend written in TypeScript. How can we get the best developer experience?

Remark: This won't be a full tutorial that shows you how to configure these libraries in your codebase. The official documentation can help you with that. Instead, consider this a conceptual overview of how to combine the technologies for the best possible developer experience.

Data Fetching with React Query

React Query, also known as TanStack Query since it's been ported to different web frameworks, has rightfully become one of the most popular React libraries. I reach for it on every project I'm involved in, either directly or through tRPC. A common myth is that React Query is a data fetching library, which it is not. It is a tool that makes asynchronous state synchronous. So you'll still need a data fetching tool. The built-in fetch browser API is the default choice. What React Query does is wrap an asynchronous fetch function and give it back to you as synchronous data through a React hook. Let's look at an example:

The traditional way to fetch data in a React component goes like this:

Initiate empty useState for data
Trigger data fetch on mount in useEffect
Store result data with setState
Saved data in state triggers re-render with data

In code form:

function DataFetchingComponent() {
  const [data, setData] = useState(null);

  useEffect(() => {
    const fetchData = async () => {
      const response = await fetch('https://api.example.com/data');
      const result = await response.json();
      setData(result);
    };

    fetchData();
  }, []);

  if (!data) {
    return <div>Loading...</div>;
  }

  return (
    <div>
      <pre>{JSON.stringify(data)}</pre>
    </div>
  );
}

With React Query we can simplify this to:

function DataFetchingComponent() {
  const fetchData = async () => {
    const response = await fetch('https://api.example.com/data');
    return response.json();
  };

  const { data, isLoading } = useQuery('data', fetchData);

  if (!isLoading) {
    return <div>Loading...</div>;
  }

  return (
    <div>
      <pre>{JSON.stringify(data)}</pre>
    </div>
  );
}

Much more elegant, isn't it? But that's not all. Now imagine the custom code you would have to write to handle the following things:

Accessing the same data in multiple components without duplicate fetches
Refreshing data during mutations
Caching and cache invalidation
Optimistic updates
Background refetching

React Query handles all of this for you! So it's not surprising that it has become a core technology in my projects. What you'll notice is that both examples don't have any TypeScript types associated. So let's fix that.

Manual API Type Safety

The previous code samples have no way of knowing the types. We fetch the data from a regular HTTP endpoint that doesn't provide any type information, both with or without React Query. A common way to make it type-safe is by setting a TypeScript type. Some projects do this with code generation, meaning there is a script that figures out the types from the backend and then generates a type definition file in your frontend. This is a valid strategy, but it requires a complex setup and build process. So a lot of projects choose to manually set the types.

// example data, could be anything
interface FetchedData {
  id: number;
  name: string;
}

function DataFetchingComponent() {
  const fetchData = async (): Promise<FetchedData> => {
    const response = await fetch('https://api.example.com/data');
    return response.json();
  };

  const { data, isLoading } = useQuery('data', fetchData);

  if (isLoading) {
    return <div>Loading...</div>;
  }

  return (
    <div>
      <pre>{JSON.stringify(data, null, 2)}</pre>
    </div>
  );
}

This works well. Your data object now has the defined type FetchedData. But there is one big problem: you cannot guarantee consistency between the response and your types. TypeScript gives you a false sense of security that a certain property will always be there. You have no way of knowing if the API suddenly no longer includes this property in the response.

Zod Validation

To ensure consistency, we need a way to validate the API response. Here, Zod comes into play. You define a schema that describes how the API response should look like:

import { z } from 'zod';

const FetchedDataSchema = z.object({
  id: z.number(),
  name: z.string(),
});

So far this looks just like our type definition, but with a slightly different syntax. But the best part is, we can automatically generate a type definition based on this schema.

type FetchedData = z.infer<typeof FetchedDataSchema>;

Now we can use this to validate the API response:

const FetchedDataSchema = z.object({
  id: z.number(),
  name: z.string(),
});

type FetchedData = z.infer<typeof FetchedDataSchema>;

function DataFetchingComponent() {
  const fetchData = async (): Promise<FetchedData> => {
    const response = await fetch('https://api.example.com/data');
    const data = await response.json();
    return FetchedDataSchema.parse(data);
  };

  const { data, isLoading } = useQuery('data', fetchData);

  if (isLoading) {
    return <div>Loading...</div>;
  }

  return (
    <div>
      <pre>{JSON.stringify(data)}</pre>
    </div>
  );
}

The parse(data) function will raise an error if the response does not match the schema. You can then handle that error however you like. Now we have the full workflow of how to call REST APIs in a type-safe manner from React using React Query and Zod. In a production app, you would move the schemas and fetch function to separate files so they can be re-used across your application. But apart from that, we have the full setup that can scale to large apps.

How to set up an Express API with Node.js and TypeScript the right way in 2024

Noah Falk — Tue, 30 Apr 2024 17:49:36 +0000

The world of Node.js servers in 2024 is a confusing one: ESM, CommonJS, TypeScript, transpiling, tsconfig, bundlers and so on. The list never ends. Even though all you want to do is set up a simple API. Unlike in the front-end world, there are no first-party CLIs that automatically bootstrap a fully configured codebase for you. Instead, you have to do it all yourself. I recently went through this myself and it took way longer than expected. So here I will be showing you the practice steps I did to get an Express.js API with TypeScript.

Prerequisites

All you need to get started is having Node.js setup in your terminal. We will be using Node v20. Our package manager of choice will be npm, which comes pre-installed with Node. But you can use any other package manager you like.

Creating the project

Create a new folder. We’ll call it express-typescript, but you can name it whatever you like. Create a package.json file inside this folder.

mkdir express-typescript
cd express-typescript/
npm init -y

This has created a package.json file with the following content. We’ll need to change a lot of it later on. But leave it for now.

{
 "name": "express-typescript",
 "version": "1.0.0",
 "description": "",
 "main": "index.js",
 "scripts": {
   "test": "echo \"Error: no test specified\" && exit 1"
 },
 "keywords": [],
 "author": "",
 "license": "ISC"
}

Creating the server

So far we only have our package.json set up. Now we need to install the dependencies we need for our server to work. Our web framework of choice is Express. It is by far the most popular framework for Node.js servers. We’ll also install two more packages: cors and dotenv. Cors will help us later on, when we create the API endpoints that need to be accessible from a browser. It will set the necessary headers, so that the browser can interact with our API. The dotenv package allows us to create a .env file in development to read environment variables from. You might know this standard from other frameworks such as Next.js.

Install the dependencies into your project:

npm install express cors dotenv

Now create a new folder called /src and add a file called server.ts inside this folder. The file should look like this. It sets up Express, Cors and a sample response to show the server is up and running.

import cors from 'cors';
import express from 'express';
import 'dotenv/config';

const port = process.env.PORT || 3000;

const app = express();

app.use(cors());

app.get('/', (req, res) => res.send('Server is running!'));

app.listen(port, () => {
 console.log(`App listening on port: ${port}`);
});

Making the types work

So far, we can’t do anything with this. We didn’t define a start command and Node.js cannot parse the TypeScript file. So we need to set up a proper build process. So let’s set up TypeScript. First off, install the necessary development dependencies:

npm i -D typescript @types/node @types/express @types/cors tsx rimraf tsc-alias

Besides TypeScript itself and the necessary typings for Node, Express and Cors we’ve also installed tsx. It’s a tool that helps us easily run our server in development. It can parse TypeScript files and automatically restarts the server whenever we make any changes. We’ve also installed rimraf and tsc-alias. These will help us later on, when we need to build our code for production.

Let’s get to configuring all of this. First off, we need a config for TypeScript. Run this command:

npx tsc --init

Now we have a tsconfig.json file in our project that we can edit. Paste this into the file, replacing all the previous content:

{
 "compilerOptions": {
   "target": "esnext",
   "esModuleInterop": true,
   "strict": true,
   "moduleResolution": "node",
   "noEmit": true,

   "outDir": "dist",

   "forceConsistentCasingInFileNames": true, 
   "skipLibCheck": true,
   "noUncheckedIndexedAccess": true,
   "noImplicitAny": true,

   "baseUrl": ".",
   "paths": {
     "@/*": ["./src/*"]
   },
 },
 "tsc-alias": {
   "resolveFullPaths": true,
   "verbose": false
 },
 "include": ["src"],
 "exclude": ["node_modules"]
}

Besides some default build-stuff, we have added additional configuration items:

An import alias so files can be imported like this @/utils/file
Configuring the tsc-alias tool so we can handle import aliases and won’t have to use .js extensions when importing files

Making it run

Now that TypeScript is configured, we need to set up the scripts to run the server. Both in development and production mode. Open package.json again and make sure the script section looks like this.

"scripts": {
  "type-check": "tsc",
  "build": "rimraf dist && tsc --noEmit false && tsc-alias",
  "start": "node dist/server.js",
  "dev": "NODE_ENV=development tsx watch src/server"
},

Here’s a quick explanation of what each script does:

type-check: checks if all TypeScript types in your project are valid
build: builds the project for production by first deleting any existing builds, then creating a new build and at the end making sure Import statements work properly
start: Start the production build
dev: Run our project in development mode.

We also need to tell Node.js that our project uses ES Modules. This is what allows you to write import instead of require.

"type": "module",

You can run any of these scripts with npm run SCRIPT. Now we’re ready to run it in development mode using npm run dev. You should see feedback in your console. Visit http://localhost:3000 to check if everything is working.

Express API endpoints

Now that we have a running server, we can set up an API endpoint. For this we’re going to be using two concepts: routers and controllers. A router will handle a certain set of sub-routes. We will create a router that handles all traffic coming to /api. Create a new file in the src directory called router.ts. Add this content:

import express from 'express';
import { getAnotherTest, getTest } from '@/controller';


export const router = express.Router();


router.get('/test', getTest);
router.get('/another-test', getAnotherTest);

You can see that we first create a new router and then add two routes to it. /test and /another-test. Both are GET methods and call a handler function inside a controller. We will create the handler functions later. First, return to server.ts and import the router. We’re using the previously created import alias for this.

import { router } from '@/router';

Then define that all requests coming to /api should go the the router we’ve just created.

app.use('/api', router);

For reference, this is how the complete file looks.

import cors from 'cors';
import 'dotenv/config';
import express from 'express';
import { router } from '@/router';

const port = process.env.PORT || 3000;

const app = express();

app.use(cors());
app.get('/', (req, res) => res.send('Server is running!'));
app.use('/api', router);

app.listen(port, () => {
 console.log(`App listening on port: ${port}`);
});

Now it’s time to create our controller. Add a new file called controller.ts with the two handler methods that return JSON:

import { Request, Response } from 'express';


export const getTest = (req: Request, res: Response) => {
 res.json({
   message: 'Hello World',
 });
};


export const getAnotherTest = (req: Request, res: Response) => {
 res.json({
   message: 'Another test route',
 });
};

Now start the server again and go to either http://localhost:3000/api/test or http://localhost:3000/api/another-test and see the JSON that is being returned.

Conclusion

With this we have finished the setup of our Express.js API with Node.js and TypeScript. I use Railway (referral) to deploy my Node servers. It’s the easiest platform I have found and very cheap to get started.

You can find the complete code for this guide on GitHub: https://github.com/noahflk/express-typescript-esm

The best free database providers in 2024

Noah Falk — Tue, 02 Apr 2024 17:06:41 +0000

A low initial cost is important for personal projects or indie hackers seeking product validation. It is hard to justify paying upwards of $20 a month just for your database, when you barely have any traffic. The other important factor is how price-effective a database can scale with you. A free tier is worth nothing when you face the choice between a huge bill or daunting provider migration as soon as your app gets any kind of traction. If you are planning to host more than a simple site project, taking the maturity of a platform into account is very important. The database is arguably the most critical link in your infrastructure. I wouldn’t want to rely on providers without a track record for apps with paid users. However, I’m more than happy to try out more exotic solutions on side projects.

Supabase - the All-in-One Solution

Supabase is a platform built on top of PostgreSQL. You can look at it as an interface for your database. So you can browse your data, look at your metrics and adjust your database settings. All in a modern user interface. Without ever touching the command line of your database.

But there’s much more to Supabase. Calling it a fancy interface on top of PostgreSQL doesn’t do it justice. It can do much more than that. It comes with built-in authentication, storage, edge functions, just to name a few. In addition, Supabase provides a neat set of libraries so you can interface with your database from anywhere. WIthout the need for a dedicated backend. A while ago I wrote a whole article about Supabase. If you want to know more, head to The good parts of Supabase.

The free tier includes:

Unlimited requests
50,000 monthly active users
500 MB database space
5 GB bandwidth
1 GB file storage

If this is not enough for you, the next tier up is the $25 a month Pro plan, which can handle most production loads. The 100,000 included monthly active users for authentication on the Pro plan are remarkable. With Auth0 that would put you squarely into the enterprise tier, costing you thousands of dollars a month.

Neon - Serverless Postgres

Neon is a serverless implementation of Postgres. This means, your database will auto-scale as needed. Even though Neon is serverless, they do not artificially limit the number of allowed requests. You just have a ressource ceiling depending on your tier. Their free tier offers:

Unlimited Databases
Unlimited requests
500 MB database space
Limited compute performance (0.25 vCPU, 1 GB RAM)

If you need more than this, there is a Launch plan priced at $19 a month. This will give you 10 GiB of storage and much more compute power, if needed.

What makes Neon special is their branching feature. You can branch your production database to apply changes, before merging it back in. For example, branching allows you to have an independent preview database for every Pull Request.

A note on Vercel Postgres

Vercel Postgres is powered by Neon, so they are essentially the same product. The pricing model is just a bit different. But there is still a free tier. And as is common with Vercel, heavy usage tends to get expensive. If you are a Vercel customer, I recommend looking into Vercel Postgres. You won’t find a simpler database to integrate into an app hosted on Vercel.

Turso - SQLite for Production

Turso is based on SQLite. Contrary to what you might think, SQLite is very capable and can most likely face any production loads you have. Plus, it offers a much better local dev experience than server-based databases such as PostgreSQL. With Turso, you do not need to connect to the server when you develop locally. Instead, you can simply have a local instance of your database on disk. So you can develop without the internet and no latency.

Since Turso is based on SQLite (technically libsql), you won’t be able to connect with database clients made for PostgreSQL or MySQL. Turso has their own library for various programming languages. However, if you’re using TypeScript I highly recommend the Drizzle Turso integration. If you’re working in an unsupported language, you can use Tursos direct HTTP access.

The most interesting feature of Turso for me is their edge support. You can easily deploy replicas all over the world, so your data is always close to your user. Keep in mind, this only makes sense if your backend runtime is also on the edge.

The free tier offers:

9 GB of storage
500 databases
3 locations (replicas)
1 billion reads
25 million writes

If you exceed storage or read/write you have the option to pay for usage instead of upgrading to the $29 a month Scaler plan. So you are not forced to commit to a high monthly fix-cost just because you slightly exceed some limit of the free tier.

Xata - the Data Platform

Xata is similar to Neon, as both offer a serverless Postgres solution with features such as branching and read replicas. In addition, both platforms have their own clients to access your database. Previously, you could not connect to Xata through the regular Postgres interface. As of right now, this is available as a beta feature.

Xata’s free tier includes 15 GB of storage. Similar to Neon, it does not explicitly limit your requests. However, you can assume that the performance of the free tier does not support very high loads. If you need more than that, you can upgrade to the usage-based Pro plan. It has a minimum charge of $20 a month, all of which goes to usage.

So what makes Xata worth mentioning here in the first place, if it’s virtually the same as Neon? They offer a lot of features on top of just being another Postgres database: file storage, full-text search and vector embeddings. If you want all of this at the same provider, Xata is the way to go.

The PlanetScale discussion

Perhaps you’ve wondered why Planetscale hasn’t been mentioned so far. They’ve recently removed their free tier. I fully understand that decision from a business perspective. They have big companies such as MyFitnessPal as customers. So gradually their target audience has shifted away from small businesses and indie hackers. Their cheapest plan now starts at $39 a month. And their website now looks very enterprise-y. Planetscale’s competitor are other enterprise solutions such as AWS RDS. The question in this realm isn’t paying $100 more or less a month for your DB. If you can eliminate the work of 2 full time staff by moving from one provider to another, you’ll happily pay thousands a month to your database provider. You’ll still come out ahead. Nothing is more expensive than human capital.

Conclusion

In wrapping up our dive into the best free hosted databases of 2024, it's pretty clear: there's no one-size-fits-all solution. Each option we explored brings something special to the table. So it all depends on your individual needs.
If you need an all-encompassing platform, Supabase is your go-to. Neon and Xata are more traditional hosted Postgres solutions, but with cool features like branching. And Turso brings SQLite to production applications with excellent edge-support.

Are there any other services that deserve to be on this list? If so, let me know in the comments!

How I get better feedback on my PRs (and how you can, too)

Noah Falk — Tue, 11 Jul 2023 11:03:15 +0000

Introduction

This article is a guide that will show you how you can set yourself up to get better feedback on every pull request, and make your development and deployment workflow a lot faster.

But before diving into how to make this happen, let’s zoom out and understand why this is so important to virtually every developer out there.

The pain of deploying changes to production

Deploying fixes and new features is a standard aspect of product development. But finding a way to ship these changes into production can be a time-consuming pain for everyone involved.

Most of the time, the journey includes multiple steps and friction points:

Initial tests and code reviews
Merge a feature branch to the main branch once these reviews are complete
Deploy a new image to a shared staging environment
Run additional tests in staging
Push the changes to production

If you are running your own infrastructure, this process can quickly become complicated and full of roadblocks: Staging servers that are blocked or out of sync. Multiple PRs opened for a single feature due to bugs found between each of these steps. And disorganized feedback from other team members that comes in at unpredictable points in the journey.

Sure, the changes get merged eventually, but the above workflow challenges create a pretty high risk of unresolved bugs and deployment delays.

Preview Environments - A partial solution

A partial solution to this problem has become increasingly popular in recent years - the use of ephemeral preview environments.

Preview environments are production-like environments that are created per commit so that the branch in question can be easily tested before it gets merged.

When introduced in a development and deployment flow, these environments enable faster reviews and more rapid feedback/approval cycles. Preview environments empower dev teams to shift the testing process to “pre-merge” where it’s easier to identify bugs and avoid the single staging-server roadblock.

But on their own, preview environments are only half the battle.

They do a great job of enabling faster reviews for every commit, but they don’t inherently help organize this feedback in an effective way. Every team member is left to the feedback mechanism of their choice, which can leave developers to sift through a disorganized mess of slack messages, emails, screenshots and shared documents.

Furthermore, setting up these preview environment mechanisms is not always so easy. And the PaaS platforms that try to streamline the process require users to use their dedicated deployment pipeline, which isn't always possible - especially in larger companies.

So to fully solve the deployment workflow dilemma, developers need a solution that is:

Easy to set up and maintain
Adaptable to cloud providers and CI pipelines
Able to effectively manage feedback from reviewers.

The solution

And this brings us to the solution that I’d like to share with you - a collaborative preview environment solution that’s easily created using two complementary tools: Pulumi and Livecycle.

Pulumi allows us to programmatically configure our infrastructure using the programming languages we already know. In our case, this will be TypeScript. They support all popular cloud providers, such as AWS or Google Cloud. For this tutorial, we’ll be using Google Cloud. There are many advantages to programmatically configuring your infrastructure. But the biggest one is: you write the config once and you can apply it as many times as you like - all with one command. And without interacting with a confusing cloud provider UI.

Pulumi supports the creation of preview environments on every pull request. They call these environments “Review Stacks” and this capability is available out of the box. With Review Stacks, Pulumi automatically provisions and deploys an ephemeral environment with our changes on our cloud provider.

And this is where Livecycle comes in. Livecyle adds a collaborative layer to each environment that is created that allows all reviewers to leave rich feedback in the context of the PR. So instead of sending confusing annotated screenshots in Slack messages or being limited to GitHub comments, Livecycle lets everyone overlay feedback directly into the existing preview environment - all in your browser, where it can be addressed quickly.

In order to see the Pulumi + Livecycle combination in action, we’ll first set up a simple static web application. Since we’re focusing on preview environments and collaboration instead of elaborate cloud deployments, we’ll use a basic HTML page as an example. Using Pulumi configuration files written in Typescript, we’ll deploy that site to Google Cloud. All without touching the UI. Afterwards, we’ll set up preview environments through Pulumi Review Stacks for all our Pull Requests on the project in GitHub. Then, we’ll hook it up to Livecycle to enable collaborative feedback in our preview environments.

So buckle up, because here we go!

Setup

Before we can get started, we need to prepare a few things. First, we need to Sign Up for a Pulumi Cloud Account. Then, install the Pulumi CLI. We strongly recommend using Homebrew on Mac for that. When you first run the CLI, you will be asked to link it with your Pulumi account. It will guide you through the process step-by-step. Also, ensure you have Node.js 14 or higher installed.

Now, we need to Sign Up for a Google Cloud account. Afterwards, create a New Project called “Pulumi Livecycle Demo”.

If your account is new, you might have to link a credit card under the “Billing” tab. Don’t worry, you should receive a free starter credit that is more than enough to run this project for months. Observe the ID pulumi-livecycle-demo on the dashboard after creating the project. Your ID might be different. But you will need it later on. So keep it handy.

For Pulumi to interact with Google Cloud from our local machine, we need to set up the Google Cloud CLI. This guide by Google illustrates how to install and set up the CLI on any platform. With that, we’re ready to write our first Pulumi configuration to deploy our site.

Deploying the site

Go to your terminal and create a new folder called pulumi-livecycle-demo. In there, run the following command to create a Pulumi project based on the official static website template for Google Cloud. It will ask you a few questions. Pick the default answer wherever available. And enter the project ID from Google Cloud we saw earlier. This is necessary so that Pulumi knows where to deploy the project to.

pulumi new static-website-gcp-typescript

Look around the project files. The www/ folder contains our HTML files that will be deployed. The index.ts file contains the Pulumi configuration to deploy our files to the web. This is the main part of the configuration:

// Import the program's configuration settings.
const config = new pulumi.Config();
const path = config.get('path') || './www';
const indexDocument = config.get('indexDocument') || 'index.html';
const errorDocument = config.get('errorDocument') || 'error.html';

// Create a storage bucket and configure it as a website.
const bucket = new gcp.storage.Bucket('bucket', {
  location: 'US',
  website: {
    mainPageSuffix: indexDocument,
    notFoundPage: errorDocument,
  },
});

We define some configuration variables and then set up a new Bucket to store the web files in. In the remainder of the file, we make this content available to everything on the web like a common web server would. Now, all that’s left to do is run pulumi up in this directory to deploy the project. You can watch how all services get created. Answer “Yes” to “Do you want to perform this update?”. Afterwards, you will see a URL where the project has been deployed to.

Setting up Automatic Deploys and Preview Environments

So far we’ve just deployed the website from our local machine. Now we want to set up a proper development workflow. Where code gets automatically deployed if you push to the main branch of your GitHub repository. And of course, we also want previews on our pull requests.

First of all, we need to add the repository we’ve been working on to GitHub. This tutorial assumes familiarity with GitHub. If you’re not sure how to do this, check out this tutorial. Now you should have the project deployed to GitHub. But nothing happens yet when we push. What we want to happen is that Pulumi triggers whenever we push code to main or open a new pull request. Pulumi Cloud can handle this out of the box.

In order for Pulumi to access our repository, we need to install their GitHub app. We can do that by visiting this link.

With that done, go to the Pulumi dashboard. On there, select our pulumi-livecycle-demo/dev stack.

Under “Settings” and “Deploy” select your repository and enable all “GitHub settings” switches according to the image below. This will tell Pulumi to deploy changes to main and create preview environments for pull requests.

However, this won’t work until we tell Pulumi about the Google Cloud credentials. Go to the project dashboard in the Google Cloud Console. Open the menu, select “IAM and admin” and “Service accounts”. Press “Create Service Account” and choose a name. Then click “Done”.

Afterward, you should see your newly created service account in the table. Click the action button and select “Manage keys”. In there, click “Add Key”, “Create new key”, and “JSON”. Download the created key. Go back to your Pulumi Stack settings and add the JSON key (in plaintext form) under the name GCLOUD_KEYFILE_JSON in the “Environments variables” section. In addition, create a new variable named PROJECT_ID and fill in the Project ID from the Google Dashboard. Your environment variables should now look like this:

With this, we’re ready to open a new pull request on GitHub! If all goes well, any changes you make will now automatically get deployed. If it fails, carefully read the error message. Chances are that the keys aren’t quite correctly configured. So carefully check if you pasted all the right keys.

Try out the deployments by pushing changes made to the www/index.html file. On your pull request you will see a comment from Pulumi with the URL of the deployed preview.

Pull Request collaboration with Livecycle

Now we get automatically created environments for every pull request. Using Livecycle we will now add collaboration to those environments. Thankfully, Livecycle is very simple to set up. So this won’t take long.

First, go to the Livecycle site, log in and click “Add a repository”. Then the wizard will guide you through the process of connecting Livecycle to your repository. Select our Pulumi Demo repository. In the next step, select “I already have a preview environment”.

Now it will tell us to create a new branch and install the SDK. Do this. We’ll add the SDK like this:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="utf-8">
    <title>Hello, world!</title>
</head>
<body>
    <h1>Hello, world! 👋</h1>
    <p>Deployed with 💜 by new <a href="https://pulumi.com/">Pulumi</a>.</p>
    <script src="https://unpkg.com/@livecycle/sdk@stable/dist/browser/index.js"></script>
    <script>
    LivecycleSdk.init().catch(console.error)
    </script>
</body>
</html>

Now there’s only one step left: telling Livecycle the URL of our preview deployment. This needs to be done on every pull request. In order to automate it, we’ll be using GitHub actions. In your repository, create a file at .github/workflows/action.yml with this content:

name: Livecycle

on:
  pull_request:

jobs:
  preview:
    name: Preview
    runs-on: ubuntu-latest
    steps:
      - name: Wait for build to succeed
        uses: fountainhead/action-wait-for-check@v1.1.0
        id: wait-for-build
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          checkName: pulumi
          ref: ${{ github.event.pull_request.head.sha || github.sha }}

      - name: link environment to livecycle
        if: steps.wait-for-build.outputs.conclusion == 'success'
        env:
          LIVECYCLE_API_KEY: ${{ secrets.LC_API_KEY }}
          PREVIEW_URL: ${{ steps.preview.outputs.value }}
        run: npx @livecycle/cli link --url=$PREVIEW_URL --from-git=. --api-key=$LIVECYCLE_API_KEY

We have to configure the LC_API_KEY in the GitHub settings. Go to “Settings”, “Secrets and variables”, and “Actions”. Select “New repository secret”. Add the key from the Livecycle site. Now you’re ready to push the changes and see if it all works!

Now we get an URL to make our reviews for each Pull Request. Using the Livecycle tools that are now included in every environment, team members can write comments, point out specific things with annotated screenshots and even record their sessions from within the PR itself. As you fix issues, you can mark them as resolved. And if something is unclear, you can ask questions directly on the issue. The collaboration platform where your team can place their feedback looks as follows:

You will also see the Livecycle feedback conveniently in the GitHub Pull Request comments:

Wrapping Up

The development workflow we’ve explored in this article offers several advantages. Using Pulumi is a great way to programmatically provision your infrastructure. This eliminates the need for manual interaction with cloud provider interfaces. Adding Livecycle to the mix enables collaborative feedback within preview environments, simplifying the process of gathering and incorporating user feedback during development.

And this combination is a powerful one for any product team. It streamlines the process of creating previews and gathering clear, actionable feedback from everyone on the team. By using Livecycle to collect feedback directly within the preview environment, teams eliminate the need for external tools or communication channels.

Upgrading your workflows with collaborative preview environments is sure to promote efficient collaboration and facilitate quicker iterations and product improvements.

I encourage you to try it out and see for yourself!

Supabase Authentication in Next.js Middleware with tRPC

Noah Falk — Thu, 11 May 2023 16:41:59 +0000

This blog post is a guide to using Supabase Authentication with tRPC in Next.js Middleware. It covers setting up Supabase Auth, creating a sign-up flow, signing in, handling forgotten passwords, and protecting routes.

Install dependencies

We'll be using the dedicated Supabase Auth Helpers for Next.js and React, as well as the Supabase JS client.

npm instal @supabase/auth-helpers-nextjs @supabase/auth-helpers-react @supabase/supabase-js

Supabase setup

If you haven’t done so already, add the Supabase Anon key to your .env file.

NEXT_PUBLIC_SUPABASE_ANON_KEY = YOUR_KEY;

Then set up the Supabase Client in src/pages/_app.tsx.

import { type AppType } from 'next/app';
import { useState } from 'react';
import { createBrowserSupabaseClient, type Session } from '@supabase/auth-helpers-nextjs';
import { SessionContextProvider } from '@supabase/auth-helpers-react';

import { trpc } from '../utils/trpc';

import '../styles/globals.css';

type Props = {
  initialSession: Session;
};

const MyApp: AppType<Props> = ({ Component, pageProps }) => {
  const [supabaseClient] = useState(() => createBrowserSupabaseClient());

  return (
    <SessionContextProvider supabaseClient={supabaseClient} initialSession={pageProps.initialSession}>
      <Component {...pageProps} />
    </SessionContextProvider>
  );
};

export default trpc.withTRPC(MyApp);

Sign Up

Now we're ready to start with the authenticiation. Create a form where you handle the signup like this.

const handleSignUp = async () => {
  const {
    data: { session },
    error,
  } = await supabaseClient.auth.signUp({
    email,
    password,
    options: {
      emailRedirectTo: process.env.NEXT_PUBLIC_APP_URL + '/auth/success',
    },
  });

  if (error) {
    // handle the error however you like

    return;
  }

  // if there is a session it means that we do not need to verify the email beforehand
  if (session) {
    router.push('/');
  } else {
    router.push('/auth/verify');
  }
};

After performing the signup we check if a session exists. If this is the case, we immediately redirect to our app. This is / in this case. If not, we show a special page /auth/verify that esentially says “Please check your email to verify your account so that you can use it”. So you'll need to create that page. The link in the email will lead to /auth/verify. This page shows a message that the user is now successfully verified and can log into the app.

Sign In

Signing in is much simpler:

const handleSignIn = async () => {
  const { data, error } = await supabaseClient.auth.signInWithPassword({
    email,
    password,
  });

  if (error) {
    // handle the error however you like
    return;
  }

  if (data) router.push('/');
};

We immediately redirect to the app in case the seemed successful.

Forgot password functionality

We need another page to deal with forgotten passwords. On there is a form where the user can enter his email address to get a reset link. The handling looks like this:

const handleForgot = async () => {
  const { data, error } = await supabaseClient.auth.resetPasswordForEmail(email, {
    redirectTo: process.env.NEXT_PUBLIC_APP_URL + '/auth/set',
  });

  if (error) {
    // handle the error however you like
    return;
  }

  // show feedback that reset email has been sent
  if (data) setSuccessMessage('Password reset email sent');
};

Then we need another form at the route /auth/set to actually set the new password. Here we get the token and then set the new password.

const handleSubmit = async () => {
  const hashArr = hash
    .substring(1)
    .split('&')
    .map((param) => param.split('='));

  let type;
  let accessToken;
  for (const [key, value] of hashArr) {
    if (key === 'type') {
      type = value;
    } else if (key === 'access_token') {
      accessToken = value;
    }
  }

  if (type !== 'recovery' || !accessToken || typeof accessToken === 'object') {
    // handle the error however you like
    return;
  }

  const { error } = await supabaseClient.auth.updateUser({
    password: password,
  });

  if (error) {
    // handle the error however you like
    return;
  }

  // go to app
  router.push('/');
};

Protecting routes

For this we use a Next.js Edge Middleware. This is the fastest way to check authentication for every page.

import { createMiddlewareSupabaseClient } from '@supabase/auth-helpers-nextjs';
import { NextResponse } from 'next/server';
import type { NextRequest } from 'next/server';

export async function middleware(req: NextRequest) {
  // We need to create a response and hand it to the supabase client to be able to modify the response headers.
  const res = NextResponse.next();

  // Create authenticated Supabase Client.
  const supabase = createMiddlewareSupabaseClient({ req, res });

  const {
    data: { session },
  } = await supabase.auth.getSession();

  // Forward to protected route if we have a session
  if (session) {
    return res;
  }

  // Auth condition not met, redirect to login page.
  const redirectUrl = req.nextUrl.clone();
  redirectUrl.pathname = '/signin';
  redirectUrl.searchParams.set(`redirectedFrom`, req.nextUrl.pathname);
  return NextResponse.redirect(redirectUrl);
}

export const config = {
  // list all the pages you want protected here
  matcher: ['/', '/dashboard'],
};

Accessing user session on the server

Create a helper like this:

import type { GetServerSidePropsContext } from 'next';
import { createServerSupabaseClient, type User } from '@supabase/auth-helpers-nextjs';

export const getUserFromContext = async (ctx: GetServerSidePropsContext): Promise<User | null> => {
  const supabaseServerClient = createServerSupabaseClient(ctx);

  const {
    data: { user },
  } = await supabaseServerClient.auth.getUser();

  return user;
};

You can now use this helper anywhere, as long as you have access to the context. Usually this would be directly in the API route:

import type { NextApiRequest, NextApiResponse } from 'next';
import { getUserFromContext } from '@/utils/serverUser';

export default (req: NextApiRequest, res: NextApiResponse) => {
  const user = await getUserFromContext(ctx);

  // do anything you want with it
};

If you’re using tRPC you can ingest it directly into your context:

import type { inferAsyncReturnType } from '@trpc/server';
import type { CreateNextContextOptions } from '@trpc/server/adapters/next';

import { prisma } from '@/server/db/client';
import { getUserFromContext } from '@/utils/serverUser';

export const createContext = async (opts: CreateNextContextOptions) => {
  const { req, res } = opts;

  const user = await getUserFromContext({ req, res });

  return {
    req,
    res,
    prisma,
    user,
  };
};

export type Context = inferAsyncReturnType<typeof createContext>;

Then you can access it in every query or mutation via ctx.user. Via this context, we can also create a protected procedure for routes that can only be called when authenticated.

// Reusable middleware to ensure users are logged in
const isAuthed = t.middleware(({ ctx, next }) => {
  if (!ctx.user || ctx.user.role !== 'authenticated') {
    throw new TRPCError({ code: 'UNAUTHORIZED' });
  }
  return next({
    ctx: {
      ...ctx,
      // infers that `user` is non-nullable to downstream resolvers
      user: ctx.user,
    },
  });
});

// Protected procedure
export const protectedProcedure = t.procedure.use(isAuthed);

Then just use this new procedure in your routers.

Supabase with TypeScript: using tRPC and Prisma to achieve end-to-end typesafety

Noah Falk — Tue, 06 Dec 2022 14:14:33 +0000

Introduction

This post picks up on an article I wrote earlier this year titled The good parts of Supabase. Wherein I reflected on my experience building Railtrack with Next.js and Supabase. If you're curious, you can read the whole post.

I started building Railtrack with plain Next.js and Supabase. I didn’t even have any Next API routes apart from the Supabase one for auth. I did all the queries client-side through the Supabase JS library.

The Supabase JS client makes it possible to build full-stack apps without any backend knowledge. Or the need to host a separate Node server. This is great for beginners and anyone else who’s trying to build a simple straightforward web app. And if this description fits you, I highly recommend starting out that way.

Limitations

As great as this approach is for small projects, it doesn’t scale that nicely. As my project grew, I ran into some limitations that started to bother me:

TypeScript experience with queries wasn’t great. This has been improved since then with Supabase JS 2.0.
No SQL query support directly from the client. Postgres stored procedure necessary.
Ephemeral database structure. You define it in the Supabase UI and not in the code.
Authorization through Row Level Security instead of the typical backend way.
Queries go straight from DB to the front end. So no possibility to process data on my backend beforehand.

For a full explanation of these things check out my previous post.

Making Supabase even more powerful with Prisma and tRPC

As you see, all of the issues I mentioned stem from querying Supabase via their JS Client and not having our own backend. Luckily that’s only one of many use cases for Supabase.

So instead, we’ll query Supabase like any other Postgres database on our own backend through an ORM called Prisma. Next.js APi routes will serve as our backend. For communication between the backend and the front end we’ll use tRPC. The combination of Prisma and tRPC gives us full type-safety between the database, backend, and front end. With the source of truth being our Prisma schema.

This more traditional setup solves all of the problems I mentioned above. And we’ll still get to benefit from the numerous other benefits Supabase offers:

Great pricing as you scale
Admin web interface to view and manage your database
Amazing auth solution, both for email/password and social auth

Guide

As a base we’re going to use the popular create-t3-app CLI to create our app with:

Next.js
Prisma
tRPC
TypeScript

It also offers Tailwind CSS and NextAuth. Feel free to add Tailwind for styling if you want. We don’t deal with authentication in this tutorial. But create-t3-app gives you the option of using Next Auth. I personally prefer using Supabase Auth directly for better support of email/password login.

Creating the project

Run this in your command line:

npm create t3-app@latest

Here are the options I chose:

Accessing the Supabase Postgres DB

Log into supabase.com and create a new project. After that, copy the Anon public key and the Postgres connection string into the .env file:

DATABASE_URL=postgres://YOUR_PASSWORD@db.YOUR_URL.supabase.co:6543/postgres
NEXT_PUBLIC_SUPABASE_ANON_KEY=YOUR_ANON_KEY

Change the database provider to postgresql and add a text property to the Example model:

// prisma/schema.prisma

datasource db {
    provider = "postgresql"
    url      = env("DATABASE_URL")
}

model Example {
  id        String   @id @default(cuid())
  text      String
  createdAt DateTime @default(now())
  updatedAt DateTime @updatedAt
}

Our database is now plugged into Prisma and ready to go. Run the following command:

npx prisma db push

Check the Supabase dashboard and you will be able to see the new table is created.

Enable Row Level Security (RLS) on that and every other table you’re going to create. But you don’t need to create any rules. What we want to do is protect the DB from being accessed publicly. But we can still access everything through Prisma with the password our server has.

Using it

As you can see, we haven’t done anything Supabase-specific in our code. We didn’t need to use their JS client for anything. But we get all the other Supbase benefits plus full end-to-end type-safety.

We can add a very simple component to fetch and display all the Example entries from our database. Add this component using <ShowExamples /> anywhere within your src/pages/index.tsx file.

In this ShowExamples component we’re going to fetch the example entries via tRPC:

import { trpc } from '../utils/trpc';

export const ShowExamples = () => {
  const examples = trpc.example.getAll.useQuery();

  if (!examples.data) return <p>Loading...</p>;

  return (
    <div>
      <ul>
        {examples.data.map((example) => (
          <li key={example.id}>{example.text}</li>
        ))}
      </ul>
    </div>
  );
};

Notice how our editor knows that example contains a text property of type string. If we were to rename this on our backend, TypeScript would notice this. It would then throw an error saying the text property no longer exists. Magical, isn’t it?

Now we need a way to add entries to our database. For this we create a second component called AddExample. Also, add that to your src/pages/index.tsx file:

import { useState } from 'react';

import { trpc } from '../utils/trpc';

export const AddExample = () => {
  const addExample = trpc.example.create.useMutation();
  const utils = trpc.useContext();

  const [text, setText] = useState('');

  return (
    <div>
      <input placeholder="name" value={text} onChange={(event) => setText(event.target.value)} />
      <button
        onClick={() => {
          addExample.mutate(text, {
            onSuccess: () => {
              utils.example.getAll.invalidate();
              setText('');
            },
          });
        }}
        disabled={addExample.isLoading}
      >
        Create
      </button>
      {addExample.isLoading && <p>Saving...</p>}
    </div>
  );
};

This does a bit more tRPC magic. Or to be more precise, React Query magic. Which powers the data fetching in tRPC.

After successfully executing the mutation to add a new example we invalidate our old query from the ShowExamples component. And any other component where we call trpc.example.getAll.useQuery(). Invalidate tells React Query that this data is now state and needs to be re-fetched immediately. We could also add optimistic updating to immediately display changes in the UI. For this, check out the docs.

Adding internationalization to a Next.js app

Noah Falk — Mon, 21 Nov 2022 19:57:04 +0000

Introduction

I usually write the UI copy in English. And no other language. But this had to change with Railtrack. It’s a web app for tracking your train journeys in Europe. With the main focus being the German-speaking countries. So it only made sense to translate the UI to German. Add the fact that I am a native German speaker, which sealed the deal.

I’m sure you know websites that define the language via the URL. So website.com/en/welcome would show the welcome page in English whereas website.com/de/welcome would show it in German. Next.js supports this out of the box. They call it internationalized routing. You define a list of supported languages. Next.js then automatically detects the user’s language set in their browser. Then it’s up to the developer (usually with the help of a library) to load the correct translations for this language.

This works great for websites. Such as marketing pages or blogs. But it’s not ideal in full-blown web apps. For two reasons:

It bloats the URL. app.website.com/dashboard looks a lot cleaner than app.website.com/en/dashboard
Language settings should be persisted on a per-account level. No matter which system the user logs in from. This means we can’t rely on the Accept-Language header alone.

How we can solve this

The flow for determining the user’s language should look like this. Ranked by priority:

If available, use the language the user has set in his settings
If no language has been set, use the default browser language

For handling the translations on the client we’ll use the excellent next-intl library. It gives us full freedom on how to fetch the translation file for the desired locale on the server.

Language choice setting

How you let the user choose his preferred language depends entirely on the specifics of your app. In Railtrack, there is a settings screen where the user can change the language. Per default, the setting will be switched to his browser language. But as soon as the user changes the setting for the first time, we’ll store the value in our database so that it can be persisted for later sessions.

Fetching the correct locale on the server

We’ll need a few things for this

Logic for deciding which local we’ll be using
Code to load translation file for current locale and sending it to the client
Consuming and displaying the translation on the client

Getting the locale

This is the logic that decides which language we’ll be using for the website. If available, the language a user has chosen in the setting is used. If none has been set, the browser language will be used.

This is handled by the following function. We’ll call it at a later point in our getServerSideProps.

// src/utils/getLocale.ts

import parser from 'accept-language-parser';
import { getCookie, setCookie } from 'cookies-next';
import { GetServerSidePropsContext } from 'next';

import { prisma } from '@/server/db/client';
import { getUserFromContext } from '@/utils/serverUser';

export const LANG_COOKIE_KEY = 'i18n-lang';
const SUPPORTED_LANGS = ['en', 'de'];
const DEFAULT_LANG = 'en';

const getBrowserLanguage = (ctx: GetServerSidePropsContext) => {
  const languages = ctx.req.headers['accept-language'];

  // use english as the default
  return parser.pick(SUPPORTED_LANGS, languages ?? '', { loose: true }) ?? DEFAULT_LANG;
};

export const getLocale = async (ctx: GetServerSidePropsContext): Promise<string> => {
  // first check for actual cookie
  const cookieLanguage = getCookie(LANG_COOKIE_KEY, ctx);

  if (cookieLanguage === 'browser') {
    // then check for browser language
    return getBrowserLanguage(ctx);
  }

  // if we have the actual cookie with a valid language
  if (cookieLanguage && typeof cookieLanguage === 'string' && SUPPORTED_LANGS.includes(cookieLanguage)) {
    return cookieLanguage;
  }

  // Only check auth after cookie fails
  const user = await getUserFromContext(ctx);

  const isAuthenticated = user && user.role === 'authenticated';

  if (!isAuthenticated) {
    return DEFAULT_LANG;
  }

  // this means we haven't set the cookie
  // so now we fetch settings and check there.
  const settings = await prisma.settings.findUnique({ where: { userId: user.id } });

  // and if it's there, store in cookie and use it
  if (settings?.language) {
    setCookie(LANG_COOKIE_KEY, settings?.language, { ...ctx, sameSite: 'lax' });

    return settings?.language;
  }

  // if we also have nothing in the DB, just use the browser language

  // set the language selection to browser from now on
  // this prevents us from having to make a DB query each time
  setCookie(LANG_COOKIE_KEY, 'browser', { ...ctx, sameSite: 'lax' });

  return getBrowserLanguage(ctx);
};

We make heavy use of cookies for caching the language preference. Why? Fetching the user settings from the database on every page load would hurt performance significantly.

Therefore we cache the result of the database query in the cookie. On the next page load, the chosen language will be extracted from the cookie without ever having to run the database query again.

Using translations

Translations for each language can be defined in simple JSON files.

// src/locales/de.json

{
  "departure": "Von",
  "arrival": "Nach",
  "add": {
    "time": "Abfahrtszeit",
    "save": "Speichern"
  }
}

// src/locales/en.json

{
  "departure": "From",
  "arrival": "To",
  "add": {
    "time": "Departure time",
    "save": "Save"
  }
}

Fetch translations in getServerSideProps on every page where you’ll use them.

export const getServerSideProps = async (ctx: GetServerSidePropsContext) => {
  // calls the function we created above
  const locale = await getLocale(ctx);

  return {
    // loads the translation for the correct locale and sends it to the page as a prop
    props: { messages: (await import(`../locales/${locale}.json`)).default, locale },
  };
};

To consume this in our components we need to add the next-intl provider to our _app.tsx file. Don’t forget to install the library: npm install next-intl.

import { AbstractIntlMessages, NextIntlProvider } from 'next-intl';
import type { AppType } from 'next/dist/shared/lib/utils';

const DEFAULT_LANG = 'en';

type Props = {
  messages: AbstractIntlMessages;
  locale: string;
};

const MyApp: AppType<Props> = ({ Component, pageProps }) => (
  <NextIntlProvider messages={pageProps.messages} locale={pageProps.locale ?? DEFAULT_LANG}>
    <Component {...pageProps} />
  </NextIntlProvider>
);

Now, using the translations on the client is as simple as adding the useTranslations hook and calling the function it returns with the desired key.

const Journeys: NextPage = () => {
  const t = useTranslations();

  return (
    <Wrapper title={t('navigation.journeys')}>
      <JourneyList />
    </Wrapper>
  );
};

Cache invalidation

Imagine user A sets the language into German on device A. He then logs in on device B where we have a cookie set to English. In that case, the implementation from above never queries the DB again to check if the language set in the cookie is still up to date. There are a few things we should do to mitigate this case:

On every login, re-fetch the language setting and update the cookie if needed. This is also important if you log in with a different user in the same browser. Because that user could have a different language setting.
Regularly refreshing the cookie information. Either by setting an expiration date on the cookie or by querying the settings DB after the page loads.

For Railtrack, I chose the strategy of invalidating the cookies on every sign in and sign up. Add the following snippet wherever you perform your sign in and sign up actions.

import { deleteCookie } from 'cookies-next';

import { LANG_COOKIE_KEY } from 'utils/getLocale';

// add this inside the login function handler
deleteCookie(LANG_COOKIE_KEY);

Wrapping up

This is what I did to add internationalization to my web app. Let me know how you like my solutions. And please reach out if you have any improvements to suggest.

If you want to see a real-world example of what I described, check out my repository for Railtrack here: https://github.com/noahflk/railtrack.

Setting up Plausible Analytics for Astro with Vercel Rewrites

Noah Falk — Wed, 16 Nov 2022 09:33:48 +0000

Plausible.io is a lightweight open-source analytics service. They offer a hosted and self-hosted option. It’s a great Google Analytics alternative that I’ve been using for quite some time now.

Setting up your application

Proxy the analytics script

Ad blockers often block Plausibles analytics script and events. This means that we won’t receive any analytics data for users with ad block enabled.

To avoid this, we will proxy all traffic to Plausible’s domain through our server. That way, ad blockers have a much harder time blocking analytics traffic. And thus, we will receive data from all our users.

To achieve this, create a file called vercel.json in your project root. Then add the following content:

{
  "rewrites": [
    {
      "source": "/plausible/js/script.js",
      "destination": "https://plausible.io/js/script.js"
    },
    {
      "source": "/plausible/api/event",
      "destination": "https://plausible.io/api/event"
    }
  ]
}

This will proxy both the loading of the Plausible embed and the API endpoint to post activity through our server.

Add the Plausible analytics script

Before you do this, you must create a project in the Plausible dashboard for this domain.

After setting up the project, add the script below to the <head> of your HTML. You can find this in the src/layouts/Layouts.astro file. Don’t forget to add your domain to the data-domain attribute.

<script defer data-domain="yourwebsite.com" data-api="/plausible/api/event" src="/plausible/js/script.js"></script>

Wrapping up

Plausible is now configured on our Astro site. From now on traffic will be captured in the Plausible dashboard. We have maximized the data we gather by proxying the data through our server.

Without the proxy, uBlock origin blocks the Plausible script on my personal site.

After adding the proxy, nothing gets blocked anymore.

The good parts of Supabase

Noah Falk — Mon, 22 Aug 2022 22:33:42 +0000

Introduction

For those that don’t know Supabase: it’s an open-source Firebase alternative. You can self-host it, but most people use the official hosting. There is a limited free tier and a $25 per project per month paid plan. That price can be steep when starting out with a small side project. But it becomes great value as the project grows.

In this article, I'll write about the experiences I had when using Supabase to build Railtrack. That’s why I’ll only talk about using Supabase for web development with Next.js with TypeScript that’s hosted on Vercel.

I’m also not going to write about the real-time feature of Supabase because I haven’t worked with it. Having said that, you might have different needs that require real-time. If that’s the case you should probably disregard much of what I say here. Instead, do your own research about the topic.

The same goes for Supabase edge functions. I deploy my sites on Vercel and use their backend hosting. So I just haven’t needed them. What I will write about are the database and authentication services of Supabase.

Query limitations

You cannot execute raw SQL queries from the Supabase client. I’m sure there are reasons for this. But I often find myself wanting to write queries that the Supabase client does not support. Or that would at least be far more efficient when using proper SQL.

The solution for this is to create a named Postgres stored procedure which you can then call from the client.

const { data, error } = await supabase.rpc("hello_world");

I don’t like this because the code for it is stowed away in the database. This means that every time I want to read or change it I need to leave my editor and go to the Supabase GUI. And it also means I now have logic inside my DB and thus outside of my codebase. Another option would be using a Postgres view. But that comes with similar drawbacks.

Ephemeral database structure

I like my database structure to be ephemeral. This means that I can tear it down and set it up again at any time (assuming there is no data). The source of truth for its structure should be in my codebase.

When using an ORM like Prisma that’s the case. All my tables are defined in the Prisma schema. All I need to do to set up a fresh DB is configure the DB connection string and run one Prisma command. Et voilà, I have a new DB with all my tables and fields.

If you’re relying on the Supabase client, that’s not possible. You don’t define a schema in your client. Instead, you set it up in the Supabase GUI or via raw SQL statements. Sure, I can write SQL scripts to define my schema. Those can be re-run whenever I need to redo the DB setup. But to me, that’s less ergonomic than having a Prisma-like schema.

The problem of going directly from DB to frontend

In real-world apps, you often want to aggregate data before sending it to the frontend. That’s what a proper backend is for. You can run intensive operations there without putting unnecessary load on the client.

In Railtrack I run complex calculations that couldn’t be done within SQL. So I need to fetch the data, run the calculations in JavaScript and then display the result.

Now assume you use the Supabase client and go directly from DB to frontend. In that case, you’d have to fetch huge amounts of data to the browser and run the calculations there just to display one calculated number. That’s not practical. So instead I have a proper backend that can run the calculations with ease.

I think Supabase edge functions could handle this use-case well. I haven’t tried them out since I’m happy with Next.js API routes hosted on Vercel.

Typesafety

The Supabase JS client in combination with TypeScript is… adventurous. There is the option to generate types from your DB schema. However, this isn’t very ergonomic since you need to use a third-party CLI tool for it. It’s worth mentioning that Supabase just announced the V2 JS client which comes with better type support. I haven’t tried it out yet so I can’t say how it compares to Prisma. What I can say is that the experience of working with Prisma is leagues ahead of the current Supabase client when it comes to types.

Authentication

Authentication is by far the greatest thing about Supabase, if you ask me. With their $25 a month plan you get 100,000 monthly active users. One hundred thousand. Auth0 doesn’t even list a price for that volume. You need to go through their enterprise sales channel. I assume you’d be paying hundreds a month for it.

The developer experience of integrating auth is pretty good too. It’s not like you only have to call one login function and boom your auth is done. There’s a bit more work to it if you want to have protected pages and access to the authed user on Next.js API routes. But everything is well documented and easy to implement.

Authorization

Per default, your tables have no protection whatsoever. We don’t want that. So we need to enable the row level security (RLS) feature. Its defaults block all access from the Supabase client.

I like how they use RLS for authorization. It’s a native Postgres feature instead of some proprietary system. But it comes with the same drawbacks we already talked about. You have to define authorization directly in your DB. And not in your source code.

I’m someone who’s used to the classic way of defining authorization. In my backend source code. So it was a big shift for me of trying to achieve the same with RLS. Don’t get me wrong. RLS is a very powerful system. It’s just very different from what I’m used to doing for authorization.

Pick the good parts of Supabase while ignoring the rest

I’m not a fan of using the Supabase client for fetching data. Yes, it may be great for beginners who just want some data in their frontend. Without running their own backend. Solutions that are so simple and try to abstract the backend rarely scale well to real-world use cases. You spend far more time creating workarounds than what the “easy” solution saves you. Another example of this is Hasura. It may look great for simple CRUD demos. But as soon as you try to solve actual problems you start having to fight its limitations.

Those critique points don’t mean Supabase is bad. Not at all. Their auth service is great. It provides arguably the best value for money of any auth service. The same goes for the database. It’s great if you treat it like any other Postgres DB. It makes far more sense to use it with something like Prisma and your own backend. You can modify your queries on the backend before sending them to your frontend. And thanks to tRPC you can establish end-to-end type safety. From DB all the way to your frontend code. That’s something the Supabase client cannot do. And that’s precisely the stack I use for Railtrack.

I also love their admin interface. It allows me to see all my tables and users directly from the browser. Without needing a clunky SQL viewer on my computer.

Automatically keep project dependencies up to date with Renovate

Noah Falk — Mon, 23 May 2022 19:54:15 +0000

We all know that keeping dependencies up to date is a tedious process. For way too long I have been doing this manually on many projects. It's a tedious process. You need to remember to periodically check all dependencies and see if there are any updates. If there are (there always are), you need to manually install and test them.

No longer. Thanks to Renovate I managed to automate most of this away. It will scan your dependencies and open pull requests for any new update. You might have heard of Dependabot by GitHub. Renovate is that but on steroids. It offers more customizations and features that allow for greater automation. This is particularly nice for those forgotten site projects that you no longer spent a lot of time but still need to be kept up to date.

Workflow

Here's what I want the tool to do for me

Updates need to be grouped together and delivered every Monday morning. This prevents me from getting daily dependency updates which can be annoying. It also keeps infrastructure cost down by only running the CI/CD pipeline once instead of separately for every single package update.
The earliest a package update should be installed is 7 days after its release. This minimizes the risk of bringing malicious code into the project before such attacks can be discovered.
Non-breaking updates (patch and minor versions) should be automatically merged into main. Provided the tests on the pull request pass.
For major updates pull requests should be opened but not grouped and automatically merged. That's because reading the changelog and manually dealing with potential breaking changes for each package is required.

Setup

To get started, add their GitHub App to your profile. Then, chose which repositories you want to use it on.

Renovate will now automatically create a pull request for you that adds a renovate.json file to your repository. You can also move this file to the .github/ subfolder if you like. However, to make Renovate behave the way we want the config must be extended.

Here’s how my renovate.json file looks like. It implements the workflow I described above.

{
  "extends": ["config:base"],
  "stabilityDays": 7,
  "prCreation": "not-pending",
  "internalChecksFilter": "strict",
  "updateNotScheduled": true,
  "timezone": "Europe/Zurich",
  "schedule": ["before 5am on Monday"],
  "packageRules": [
    {
      "matchPackagePatterns": ["*"],
      "matchUpdateTypes": ["minor", "patch"],
      "groupName": "all non-major dependencies",
      "groupSlug": "all-minor-patch",
      "labels": ["dependencies"],
      "automerge": true
    },
    {
      "matchPackagePatterns": ["*"],
      "matchUpdateTypes": ["major"],
      "labels": ["dependencies", "breaking"],
      "automerge": false
    }
  ]
}

I don't enable automatic merge on important projects I actively work on. On those projects it's worth it for me to quickly go through non-breaking dependency updates and merge them manually if they're okay.

Conclusion

After commiting the updated renovate.json settings, your automated dependency updater is complete.

Now we have an automated but worry-free approach to managing your project dependencies. That's how it looks like:

Update for a package comes out
Every Monday morning new updates get applied in a new pull request
Your automated test suite for pull requests runs
If all tests pass, non-breaking changes get automatically merged into main
The changes get automatically deployed to production

4 tips to supercharge your Git experience

Noah Falk — Mon, 17 Jan 2022 15:02:04 +0000

Having recently done the work of setting up my new computer from scratch, I think I've landed on a pretty sweet Git setup. Let me share it here.

The tips in this tutorial require changes to the Git configuration file. This file can be found in your home directory as .gitconfig on any OS. I personally use Mac so some things in here might be different on other platforms.

Use main as default branch

In 2020 GitHub among many others changed the default branch name for new repositories from master to main. Doing the same on our local machines can be achieved with one setting

# ~/.gitconfig

[init]
    defaultBranch = main

Auto sign commits

Do you want your commits on GitHub to have a green "Verified" badge? ✅ This ensures that you are indeed the author of the commits and that they have not been tampered with (which is surprisingly easy to do by the way).

First, you need to generate a GPG key pair. This is very similar to SSH keys, which you will most likely already have. If you use a Mac, GPGTools is the best tool I could find to generate and manage GPG Keys.

Now, export the public key through GPGTools and upload it to your GitHub user account.

After that we need to tell Git to sign all commits with our new key.

# ~/.gitconfig

[user]
    name = Noah
    email = hi@noahflk.com
    singingkey = <your key signature>

[commit]
    gpgsign = true

There is a great explanation here if you need more info.

Multiple Git identities

If you're like me and need to use different email addresses for work and private projects, you can automatically adjust your identity based on which directory you're in.

And yes, you can even set different GPG keys for both your work and private email addresses.

First you will need to define the identity that applies everywhere per default in your main configuration.

# ~/.gitconfig

[user]
    name = Noah
    email = hi@noahflk.com
    singingkey = <your key signature>

Then, create a second file for your work identity and import it in the main configuration only if you are in a certain directory.

# ~/.gitconfig

[includeIf "gitdir:~/Repos/work/"]
        path = .gitconfig-work

# ~/.gitconfig-work
[user]
    name = Noah
    email = supersecretwork@email.com
    singingkey = <your key signature>

Now, make sure you clone all your work repositories into the directory you specified. All repositories in there will automatically make use of your work identity. Any repository not in there will use the default identity.

Only push the current branch

By default Git will push all branches to the remote and merge them. This is a matter of preference but I do not like it. Therefore I have Git configured so that per default only the current branch that is tracking a remote upstream branch will be pushed. See a more detailed explanation here.

# ~/.gitconfig

[push]
    default = simple

Conclusion

These are some of the tricks I use to improve my day-to-day Git/GitHub experience. Let me know if you have any other suggestions!

If you liked this you can connect with me on Twitter

How to use the new Next.js Image Component in your MDX Blog

Noah Falk — Mon, 16 Nov 2020 18:31:35 +0000

Update Summer 2021

Vercel has now released released Next.js 11 which once again brings improvements to image optimization. So manually specifying the image size is no longer necessary, as long as the image is directly imported:

import Image from "next/image";
import picture from "/profile-picture.jpg";

export default function Home() {
  return <Image src={picture} alt="Profile Picture" />;
}

So, as long as your MDX engine supports importing React components in MDX files, all you need to do to benefit from image optimization is this:

// blog/blog-post.mdx
import Image from "next/image";
import picture from "/profile-picture.jpg";

# Title
Some Markdown

<Image src={picture} alt="Profile Picture" />

Introduction

In October Vercel released Next.js 10 which includes a component for automatic image optimization. This component decreases loading times by using a couple of smart techniques, like dynamically serving the image in the size it gets rendered at or lazy-loading images.

Vercel advertises the next/image component as a drop-in replacement for the HTML <img> element.

<img src="/profile-picture.jpg" width="400" height="400" alt="Profile Picture">

Gets replaced with

import Image from 'next/image'

<Image src="/profile-picture.jpg" width="400" height="400" alt="Profile Picture">

next/image requires the developer to specify a width and height for the image to avoid Cumulative Layout Shift. When you don't specify with and height attributes on an image the browser won't know how much space the image will take up until the image has loaded. This causes the content below the image to change position because space is needed to display the loaded image.

Source

Status quo

The articles on my blog are written in MDX and Next.js Static Site Generation (SSG) renders them at build time. Webpack's file-loader makes it possible to have images in the directory of that specific article and import them like any regular file. They then get moved into the public Next.js folder at build-time and the public URL gets returned with the import statement.

// Import image like any source code file
import progressBar from "./progress-bar.png";
// progressBar -> /static/media/progress-bar.b95cd6c9d38c5d8c614c8946d1c36baf.png

// Usage
<img src={progressBar} alt="Progress bar without any labelling" />;

The MDXProvider from @mdx-js/react then maps all <img> elements in MDX files to the <Image> component form Chakra UI to apply some styling.

import { Image } from "@chakra-ui/core";

const components = {
  img: (props) => <Image rounded="lg" {...props} />,
};

export default components;

My Webpack config for file-loader looks as follows:

config.module.rules.push({
  test: /\.(png|jpe?g|gif|mp4)$/i,
  use: [
    {
      loader: "file-loader",
      options: {
        publicPath: "/_next",
        name: "static/media/[name].[hash].[ext]",
      },
    },
  ],
});

Migrating to next/image

Since next/image works as a drop-in replacement, all I'd need to do is change the <img> elements in every MDX file the next/image component and add width and height props. But this would mean manually editing every single MDX file and determining the resolution of every single image. So instead I decided to find a way to do this automatically. Since Next.js blogs use static site generation you can easily run a script at build time which detects the resolution of any image.

Since I was already using file-loader I decided to modify it a little bit to be able to run some custom code for every imported image, so I can detect its resolution. The return value of this callback method can be accessed through the import statement in the React code. I published this modified version of file-loader on NPM under the name enhanced-file-loader.

This is my updated Next.js config where I call a function from the image-size package to determine the width and height of my images.

const sizeOf = require("image-size");

config.module.rules.push({
  test: /\.(png|jpe?g|gif)$/i,
  use: [
    {
      loader: "enhanced-file-loader",
      options: {
        publicPath: "/_next",
        name: "static/media/[name].[hash].[ext]",
        generateMetadata: (path) => {
          return sizeOf(path);
        },
      },
    },
  ],
});

I decided to tweak my MDX files a little bit for simplicity's sake. However, this is optional since you could theoretically route everything through the src prop on the <img>.

// Import image like any source code file
import progressBar from "./progress-bar.png";
// progressBar:
// { url: "/static/media/progress-bar.HASH.png", metadata: { width: 1000, height: 5000 } }

// Usage
<img {...progressBar} alt="Progress bar without any labelling" />;

And finally, in the file where I map tags from MDX to React components, I can use the next/image component.

import { Image } from "@chakra-ui/core";

const components = {
  img: (props) => <NextImage width={props.metadata.width} height={props.metadata.height} {...props} />,
};

export default components;

Conclusion

This is everything I did to add Image Optimization to my Next.js blog. It makes it possible to add new images with zero additional effort while still enjoying the benefits of this new Next.js feature. However, this is simply how I chose to implement this. I am fairly new to both Next.js and customizing Webpack, so take my advice with a grain of salt. There might very well be a way better way to do this. Be sure to let me know if you found another (perhaps better) way to do this.

DEV Community: Noah Falk

Making a REST API typesafe with React Query and Zod

Data Fetching with React Query

Manual API Type Safety

Zod Validation

How to set up an Express API with Node.js and TypeScript the right way in 2024

Prerequisites

Creating the project

Creating the server

Making the types work

Making it run

Express API endpoints

Conclusion

The best free database providers in 2024

Supabase - the All-in-One Solution

Neon - Serverless Postgres

Turso - SQLite for Production

Xata - the Data Platform

The PlanetScale discussion

Conclusion

How I get better feedback on my PRs (and how you can, too)

Introduction

The pain of deploying changes to production

Preview Environments - A partial solution

The solution

Setup

Deploying the site

Setting up Automatic Deploys and Preview Environments

Pull Request collaboration with Livecycle

Wrapping Up

Supabase Authentication in Next.js Middleware with tRPC

Install dependencies

Supabase setup

Sign Up

Sign In

Forgot password functionality

Protecting routes

Accessing user session on the server

Supabase with TypeScript: using tRPC and Prisma to achieve end-to-end typesafety

Introduction

Limitations

Making Supabase even more powerful with Prisma and tRPC

Guide

Creating the project

Accessing the Supabase Postgres DB

Using it

Links

Adding internationalization to a Next.js app

Introduction

How we can solve this

Language choice setting

Fetching the correct locale on the server

Getting the locale

Using translations

Cache invalidation

Wrapping up

Setting up Plausible Analytics for Astro with Vercel Rewrites

Setting up your application

Proxy the analytics script

Add the Plausible analytics script

Wrapping up

The good parts of Supabase

Introduction

Query limitations

Ephemeral database structure

The problem of going directly from DB to frontend

Typesafety

Authentication

Authorization

Pick the good parts of Supabase while ignoring the rest

Automatically keep project dependencies up to date with Renovate

Workflow

Setup

Conclusion

4 tips to supercharge your Git experience

Use main as default branch

Auto sign commits

Multiple Git identities

Only push the current branch

Conclusion