DEV Community: ktmouk

9 Tips for Building a Secure Web Application with tRPC, Next.js, Prisma, Turbo, and NextAuth

ktmouk — Fri, 26 Jul 2024 14:47:28 +0000

I recently made Minute: an open-source time tracking application for individuals using the T3 Stack as a weekend project. In this article, I'll introduce some security tips that I learned while making Minute.

Directory Structure

This section is not directly related to security but is explained first to make the following sections easier to understand.

I adopted Turborepo to manage my source code as a monorepo. Given making a monorepo project, I think the most difficult part is how to separate the source code and what directory structure to use.

At first, I considered creating a directory structure based on Clean Architecture, but after starting, I changed my mind. This is because, while Clean Architecture is a well-known and well-thought-out approach, it is a bit over-engineered for Minute.

I eventually simplified the directory structure like this:

I will explain some packages, except for those that are obvious.

`prisma/`

GitHub Link

This package includes the Prisma schema file, migration files, and Prisma Client. Notably, Prisma Client is imported from multiple packages: trpc, service, and apps/web. I recommend creating a prisma/ package as it makes it easier to understand which files are related to Prisma.

`schema/`

GitHub Link

This contains the Zod schemas for each Prisma model and some constants related to validation, such as MAX_NAME_LENGTH, REGEXP, etc. These are used for validation.

`services/`

GitHub Link

This contains functions that execute a single procedure, such as inserting or updating a model after validating parameters.

`trpc/`

GitHub Link

This includes tRPC middleware, routers, and procedures. All procedures contain no logic and simply delegate processing to the appropriate service. By doing this, I can easily switch to another library (e.g., serverAction) in the future without changing the logic, if needed.

Validate Parameters in Each Package

Validation is the most important part of protecting a web application. Therefore, I wanted to validate input parameters passed by users and outputs in both the trpc and service packages. This approach is similar to Contract Programming.

In Contract Programming, every entry point (package) should validate input/output values, and if a validation fails, the function throws an error and does not execute the remaining process for security reasons.

Fortunately, tRPC already has the capability to validate both inputs and outputs, Zod library also has a similar feature named z.function()

I thus made a helper function named contract, which is a tiny wrapper of z.function().

import type { ZodTypeAny } from "zod";
import { z } from "zod";

export const contract = <
  I extends ZodTypeAny,
  O extends ZodTypeAny,
  F extends (input: z.infer<I>) => z.infer<O>,
>(
  { input, output }: { input: I; output: O },
  implement: F,
) => {
  return z.function().args(input).returns(output).implement(implement);
};

And use it in each service package.

export const getUser = (db: PrismaClient) =>
  contract(
    {
      // the userId should be UUID otherwise it throws an error.
      input: z.strictObject({
        userId: z.string().uuid(),
      }),
      // the result object should be the user object.
      output: z.promise(userSchema),
    },
    async (input) => {
      const user = await db.user.findFirst({
        select: {
          id: true,
          name: true,
          image: true,
          createdAt: true,
          updatedAt: true,
        },
        where: {
          id: input.userId,
        },
      });
      if (user === null) {
        throw Error("The user does not exist.");
      }
      return user;
    },
  );

The trpc package also validates input/output data.

export const tasksRouter = router({
  getTasksInFolder: protectedProcedure
    .input(
      z.strictObject({
        folderId: z.string().uuid(),
      }),
    )
    .output(z.array(taskSchema))
    .query(async ({ input, ctx }) => {
      return getTasksInFolder(ctx.db)({
        userId: ctx.currentUserId,
        folderId: input.folderId,
      });
    }),
...
})

I love this approach. Notably, output validation is as important as input validation. For example, imagine we accidentally added sensitive data (e.g., sessionToken) into a result object. Output validation raises an error instead of passing the result to a user.

Setting Custom tRPC Error Formatter

Since tRPC provides a default error formatter, you can use tRPC without having to set up a custom error formatter. However, I recommend setting a custom error formatter.

If an error is thrown, tRPC will return an error.message as a response by default. This is usually not a problem, but some libraries add verbose messages to the error.message when an error occurs.

For example, the Zod library adds a helpful message to error.message.

This message is useful when debugging, but there is a risk that an attacker can guess what errors have occurred internally. I thus recommend setting a custom formatter like this:

initTRPC.context<typeof createInnerContext>().create({
  transformer: superjson,
  errorFormatter({ shape }) {
    // Hide a message to avoid exposing server errors.
    return { ...shape, message: "" };
  },
});

By doing that, tRPC always returns an empty string if an error occurs.

Manage ENVs with T3 Env

T3 Env is used to manage application environment variables type-safely. Since it is rare for an application not to have any environment variables, I recommend this library to everyone who uses Next.js.

If you are going to use this, I also recommend to split schemas into two files: one for public (client) and one for secret (server), according to this warning in the official docs.

This is because validation schemas (keys) for the server variables will be shipped to the client if you don't separate the schema. Of course, ENV values are not exposed, so it's not mandatory. However, validation schemas can provide hints to attackers about what kind of libraries the application uses internally (e.g. NEXTAUTH_***).

Disable fetchCache If You Don't Use It

When creating Minute, an open-source time tracking application, I decided to make API requests on the client side and not make requests on SSR (RSC) since all pages handle private data and require authentication, and I wanted to avoid worrying about caching issues.

If you want to disable caching fetched data, set only-no-store or force-no-store to fetchCache on the page.

export const fetchCache = "only-no-store";

When force-no-store is passed, Next.js does not cache fetched data even if you provide a force-cache option on child components. In contrast, only-no-store also disables caching and throws an error if you provide a force-cache option.

I personally recommend using only-no-store because it alerts you if you or someone else enables caching inadvertently, allowing you to notice the mistake.

Error: cache: 'force-cache' used on fetch for https://... with 'export const fetchCache = 'only-no-store'

Add The CSP Header

The CSP header is essential for mitigating the risk of XSS attacks. Fortunately, here are the instructions on how to set up the CSP header in the Next.js docs.

Apart from the CSP Header, I recommend adding other security headers. There are libraries specifically for adding these headers (e.g next-safe).

Add CSRF prevention

If your Next.js application has API routes, adding Origin header validation in middleware is worthwhile to prevent CSRF attacks. Modern browsers include the origin that caused the request in the Origin header before sending an API request, which can be used to determine whether the request is from the same origin.

const hasValidOrigin = (req: NextRequest) => {
  const origin = req.headers.get("Origin");
  return origin === null || origin === 'https://YOUR-ORIGIN';
};

FYI: This technique is also used in ServerAction.

For those using tRPC, it is also recommended to add validation to ensure that the Content-Type is JSON. This approach will be implemented in tRPC v11, but since v11 is currently in beta, you can add this validation by implementing it yourself.

const hasJsonContentType = (req: NextRequest) => {
  const contentType = req.headers.get("Content-Type");
  return (
    typeof contentType === "string" &&
    contentType.startsWith("application/json")
  );
};

Additionally, if you are using cookies to store session IDs, I recommend checking the HttpOnly, Secure, and SameSite attributes of the cookie. (I believe NextAuth sets these attributes correctly by default, though.)

Add the Cache-Control header

The Cache-Control header is used to control whether the response data can be cached or not. In the case of Minute, all tRPC responses are private and I don't want to cache them.

To add the Cache-Control header, simply add the setting to next.config.js.

/** @type {import('next').NextConfig} */
const nextConfig = {
  reactStrictMode: true,
  headers() {
    return [
      {
        source: "/api/:path*",
        headers: [
          {
            key: "Cache-Control",
            value: "private, no-store, no-cache, must-revalidate",
          },
          { key: "Pragma", value: "no-cache" },
        ],
      },
    ];
  }
}

Note that you cannot set Cache-Control headers in next.config.js for pages or assets. Next.js automatically adds appropriate Cache-Control headers to each page (maybe, I hope).

Use server-only

In your Next.js project, there are files meant to be used only on the server, and you may not want to expose them to the client's browser. In that case, you can use the server-only package.

If a file imports server-only packages and is imported from the client, throw an error to prevent leaking server-side code. I recommend adding an import of server-only to every file that obviously does not need to be used by the client. For example, in [...nextauth]/route.ts):

// Prevent importing from client-side code.
import "server-only";

import NextAuth from "next-auth"

const handler = NextAuth({
  ...
})

export { handler as GET, handler as POST }

Test Backend Code

Backend tests are essential for keeping your application healthy. In the case of Minute, since I used Prisma in the backend (trpc and service package), I had to mock the Prisma Client or use a Docker container to set up a database for testing to write the backend tests. The Prisma docs introduce both options: Unit testing and Integration testing. I adopted Integration testing because it is more accurate than Unit testing as it uses a real database.

If you are planning to write integration tests, you may want to reset the database every time the test runs. In that case, the --force-reset option would be helpful.

// package.json
{
  ...
  "scripts": {
    "db:reset": "prisma db push --force-reset"
  }
}

Also, if you are struggling to create dummy data for testing, I recommend FactoryJS, which is for simply creating objects or Prisma records.

When you write tests, I recommend covering both common cases and edge cases. For example, what if the given ID is invalid?

describe("deleteChart", () => {
  describe("when a user has the chart", () => {
    it("deletes the chart and its items", async () => {
      const user = await userFactory.create();
      const chart = await chartFactory.vars({ user: () => user }).create();
      await expect(
        deleteChart(db)({
          id: chart.id,
          userId: user.id,
        }),
      ).resolves.toBeUndefined();
      await expect(
        db.chart.findFirst({
          where: {
            id: chart.id,
          },
        }),
      ).resolves.toBeNull();
    });
  });

  describe("when the id is invalid", () => {
    it("throws an error", async () => {
      const user = await userFactory.create();
      await expect(
        deleteChart(db)({
          id: 'INVALID',
          userId: user.id,
        }),
      ).rejects.toThrow("The chart does not exist.");
    });
  });
});

Conclusion

I introduced some tips that I found during the development of Minute. However, I'm not sure if I've listed all the important tips (probably not, as these libraries are complicated and change day by day). But at least I believe this article will be helpful to everyone who is going to make an application with Next.js, Prisma, etc.

I hope this article is helpful and interesting to you!

If you are interested in Minute, you can access the repository from the link below. We also welcome contributions.

ktmouk / minute

⏰ The open-source time tracking app for individuals.

minute

The open-source time tracking app for individuals.

About

Minute is an open-source time tracking app.
In contrast to other time tracking apps, Minute is mainly focused on individual use and is designed to help users review how they spend their time and use it more meaningfully.

Demo

Give Minute a try at the demo site! — no account registration required:

👉 Visit Demo

Please note that the demo site may be closed without notice.

Screenshots

Features

Folders

Manage your tracked time entries with folders and analyze your recent time usage per folder on the report page. You can also view and edit created folders and time entries anytime from the sidebar.

FactoryJS - the TypeScript version of factory_bot

ktmouk — Wed, 08 May 2024 13:26:05 +0000

Recently, I've been developing a web app in TypeScript and struggled with writing clean tests. This article introduces factory-js and how to use this library.

What is factory_bot?

factory_bot is a popular Ruby library for generating dummy models for tests.
Here is a simple example demonstrating how to use factory_bot:



def bio(user)
  "I'm #{user.name}"
end

# 1. Define the user factory
factory :user do
  name "john"
  email { "#{name}@example.com" }
end

describe "#bio" do
  it "returns bio" do
    # 2. build the user model with defined factory.
    user = build(:user, name: "james")

    # 3. Write tests using the generated model.
    expect(bio(user)).to eq("I'm james")
  end
end

factory_bot has many advantages:

Time-saving: Writing multiple lines of code just to create models is tedious. factory_bot can create models with almost a single line of code.
Maintainability: Once a factory is defined, it can be reused across multiple tests. For instance, if you have to add new required columns to the user table, you just need to add new attributes to the user factory.
Readability: factory_bot offers useful features like traits and dependent attributes, enhancing the readability of tests.

In fact, I use factory_bot in many of my projects, and it has become an indispensable library for writing highly maintainable tests.

What about the TypeScript world?

These days, you can use well-crafted ORM libraries such as Prisma, Drizzle, MikroORM, etc.

However, what about a testing library similar to factory_bot for TypeScript?

I have recently been developing an API for a hobby project in TypeScript and was looking for a library like factory_bot.

Requirements

Ideally, I was looking for a factory library that met the following requirements.

Provide as much of the same features as factory_bot as possible (e.g. traits) while ensuring type safety.
Support multiple ORM libraries (e.g. Prisma, Drizzle).
Capable of creating models with complex relations: e.g. N:1, N:M, STI, self-relationships.
Supports type inference from objects, eliminating the need to define types manually.

After an extensive search, I could not find a factory library that met these requirements, so I decided to create my own.

Is it really possible?

First, I looked into what APIs the popular ORMs have, and found that the APIs differ from ORM to ORM.

For example, Prisma returns a plain object, while TypeORM returns an instance of the class. Additionally, Prisma supports connects, which allows related models to be created simultaneously—a feature that DrizzleORM does not have.

These differences were enough to make my brain explode 🤯. I was wondering which ORM to design factory-js based on.

Ultimately, I realized that an ORM essentially acts as a wrapper for an SQL builder and is based on the CREATE TABLE DDL. A relational database (RDB) table basically consists of two elements: columns and foreign keys (although there is also a primary key, it is not crucial for factory-js).

Consequently, I added two features to factory-js: props and vars. props corresponds to columns and vars corresponds foreign keys.

API design

As shown in the following picture, the factory-js API is almost identical to the factory_bot API and is intuitive.

props

props is a property of the object, corresponding to the columns of the table.



const userFactory = await factory
  .define({
    props: {
      firstName: () => faker.person.firstName(),
      lastName: () => faker.person.lastName()
    },
    vars: {},
  })

const user = userFactory.build();
console.log(user) // { firstName: "...", lastName: "..." }

Each property value is wrapped in a function so that random generator libraries such as faker can be used.

It is also possible to override default property values in the method chain. This is useful when tests need to use fixed values rather than randomly generated values.



it("returns firstName", async () => {
  const user = userFactory
    .props({ firstName: () => "John" })
    .build()

  expect(user.firstName).toBe("John")
})

vars

This is to define variable and similar to factory_bot's transient. In contrast to props, variables are not added as properties to the built object.

Unfortunately, unlike factory_bot, variables cannot be used in the .define method but are usable in the method chain after the factory has been defined.



const userFactory = await factory
  .define({
    props: {
      // Set a temporary value so that TypeScript can infer types
      name: () => "",
    },
    vars: {
      title: () => "Mr.",
    },
  })
  .props({
    // Then, use a variable to set the actual value.
    name: async ({ vars }) => `${await vars.title} John`,
  });

const user = await userFactory.build();
console.log(user); // { name: 'Mr. John' }

This is due to a limitation of TypeScript: factory_bot does not have a type, but factory-js does, and factory-js infers types based on a object passed to .define. TypeScript tends to fail to infer the type if variables can be used in .define, because vars and props can have circular dependencies.

create

Factory-js also has .create method, which is similar to factory_bot. In contrast to .build, this method inserts records into the database, so you can write tests that require stored records.

To use create, you need to pass a function as the second argument that insert a record into the database with the ORM you are using.



const userFactory = await factory.define(
  {
    props: {
      firstName: () => "John",
      lastName: () => "Doe",
    },
    vars: {},
  },
  // This implementation depends on the ORM.
  async (props) => await db.user.create({ data: props }),
);

// Returns a saved object.
await userFactory.create();

Examples

As explained in the previous section, we can define factories that have associations using only props and vars features. For example, the following factories have 1:1 relationships. A profile model belongs to a single user model.



const userFactory = factory.define(
  {
    props: {
      id: () => faker.number.int(),
      role: () => 'admin',
    },
    vars: {},
  },
  async (props) => await db.user.create({ data: props })
)

const profileFactory = factory.define(
  {
    props: {
      id: () => faker.number.int(),
      userId: () => faker.number.int(),
      firstName: () => faker.person.firstName(),
      lastName: () => faker.person.lastName()
    },
    vars: {
      user: async () => await userFactory.create(),
    },
  },
  async (props) => await db.profile.create({ data: props })
).props({
  userId: async ({ vars }) => (await vars.user).id
});

In the example above, vars is used to create the associated user model. The process is as follows:

Let's compare with CREATE TABLE DDL. Where props corresponds to columns and vars to foreign keys.

Is vars really required?

In fact, the use of vars is not mandatory in the previous example. It is also possible to define a profile factory without the use of vars, as shown below:



const profileFactory = factory.define(
  {
    props: {
      id: () => faker.number.int(),
      // create user model and set its id.
      userId: async () => await userFactory.create().id,
      firstName: () => faker.person.firstName(),
      lastName: () => faker.person.lastName()
    },
    vars: {},
  },
  async (props) => await db.profile.create({ data: props })
)

However, RDB tables sometimes have multiple foreign keys, for example, the following user table has composite primary key: firstName and lastName. In this case, we have to use vars to refer to the same user.



const userFactory = factory.define(
  {
    props: {
      id: () => faker.number.int(),
      firstName: () => faker.person.firstName(),
      lastName: () => faker.person.lastName()
    },
    vars: {},
  },
  async (props) => await db.user.create({ data: props })
)

// BAD
const profileFactory = factory.define(
  {
    props: {
      id: () => faker.number.int(),
      // ❌ The user relationship cannot be created correctly
      //    because firstName and lastName
      //    refer to different users.
      firstName: () => (await userFactory.create()).firstName,
      lastName: () => (await userFactory.create()).lastName,
    },
    vars: {},
  },
  async (props) => await db.profile.create({ data: props })
)

// GOOD
const profileFactory = factory.define(
  {
    props: {
      id: () => faker.number.int(),
      firstName: () => '',
      lastName: () => ''
    },
    vars: {
      user: async () => await userFactory.create(),
    },
  },
  async (props) => await db.profile.create({ data: props })
).props({
  // ⭕ This works correctly because
  //    these columns refer to the same user.
  firstName: async ({ vars }) => (await vars.user).firstName,
  lastName: async ({ vars }) => (await vars.user).lastName,
});

Since it is tedious to use vars and props depending on whether it is a single or multiple foreign key, it is generally recommended to use vars.

How does it work inside?

Factory-js uses two simple tricks internally to implement vars and props: the Proxy and a simple memo function.

Proxy enable you to intercept the property access and customize returns values. Proxy is also used in Vue3.

The memo function is a small closure for caching calculated values so that vars returns the same value no matter how many times it is called.



export const memo = <V>(func: (...args: any) => V) => {
  let evaluated = false;
  let value: V | undefined = undefined;

  return (...args: A) => {
    // If this is the first call,
    // Execute the `func` and stores the result to `value`.
    if (!evaluated) value = func(...args);
    evaluated = true;
    return value;
  };
};

As you can see from bundlephobia, factory-js is currently a very small package because it contains very simple logic.

Other features

Other than vars and props, factory-js has more powerful features that inspired by factory_bot.

trait

I think the most useful feature of factory_bot is trait, and factory-js also supports trait.

trait enable you to manage multiple props, vars with a key and apply them with .use.



const userFactory = await factory
  .define({
    props: {
      role: () => "guest",
      isAdmin: () => false,
    },
    vars: {},
  })
  .traits({
    admin: {
      props: {
        role: () => "admin",
        isAdmin: () => true
      }
    }
  })

const user = await userFactory.use((t) => t.admin).build();
console.log(user) // { role: "admin", isAdmin: "true" }

dependent props / vars

This is the most powerful feature of factory-js. Thanks to Proxy, you can define properties that depend on other properties.

The following example compute the fullName value from firstName and lastName.



const user = await factory
  .define({
    props: {
      firstName: () => "John",
      lastName: () => "Doe",
      fullName: () => "",
    },
    vars: {},
  })
  .props({
    fullName: async ({ props }) =>
      `${await props.firstName} ${await props.lastName}`,
  });

const user = await userFactory
  // dependent props works correctly
  // even if the prop value is changed later.
  .props({ firstName: () => 'Tom' })
  .build();

// { firstName: 'Tom', lastName: 'Doe', fullName: 'Tom Doe' }
console.log(user);

Are there really cases where this functionality is needed? In fact, I have experienced cases where I want to use dependent props to avoid validation errors.

For example, suppose an event log model has startTime and endTime columns and this model has a validation that checks if startTime is before endTime. In this case, you can use this feature to set endTime to one second before startTime to avoid validation errors.

Prisma Plugin

As new features are added to the product, the database tables grow daily. I didn't want to define factories myself each time a new table was added.

So I created a prisma plugin for factory-js. The plugin generates factories based on the prisma schema file.

Factory-js only has a prisma plugin at the moment, but I think it would be possible to implement plugins for other ORMs such as DrizzleORM.

Conclusion

So far I'm very happy with factory-js and am able to write clean tests with it. If you are interested in factory-js, you can try factory-js as it is open source.

factory-js / factory-js

🏭 The object generator for testing.

FactoryJS

The object generator for testing.

Documentation・Contributing Guide

Features

✅ Type Safety - It helps prevent configuration errors and typos.
⚙️ Highly Functional - You can define variables, traits, and properties that depend on other properties.
🪶 Lightweight - It is very lightweight as it does not depend on any other packages.
📦 ORM-Friendly API - Designed to be used also with ORMs like Prisma and Drizzle.

Quick Start

Install FactoryJS.

pnpm add -D @factory-js/factory

Define a factory and use it in your tests, database seeds, etc.

import { factory } from "@factory-js/factory";
// Define the factory
const userFactory = factory.define({
  props: {
    firstName: () => "John",
    lastName: () => "Doe",
    role: () => "guest",
  },
  vars: {},
});

describe("when a user is admin",

…

View on GitHub

I hope this article will interest and help you! 💖

Using Vue 2 component in Astro, not Vue3 .... is it possible?

ktmouk — Sat, 11 Feb 2023 10:06:02 +0000

Introduction

Recently, I've become interested in Astro and Island Architecture and have started reading its documentation.

As you know, Astro has a great feature called UI Framework Integrations that allows you to write components using popular frameworks such as React!

According to the official documentation, Astro already supports almost all of the popular frameworks: React, Vue3, and Svelte, which is great...... wait, where is Vue2?

Vue2 will reach EOL at the end of 2023

Seriously, I know that it has been announced in the official docs that Vue2 will be EOL, so there is no way Astro will support Vue2 in the future.

However, I am curious if it is technically possible for Astro to support Vue2.

Let's make Vue2 integration myself!

Fortunately, Astro's documentation already covers how to create a new integration and is easy to understand. And surprisingly, only three files need to be created to add a new integration: main.js, client.js, and server.js!

Now let's look into how to create a new integration.

main.js

This file is a sort of configuration file for the integration. When the astro:config:setup hook is called, Astro will add a renderer and update Vite settings according to this file.

import { AstroIntegration } from "astro";
import { createVuePlugin } from "vite-plugin-vue2";

function getRenderer() {
  return {
    name: "astro-vue2",
    clientEntrypoint: "astro-vue2/client.js",
    serverEntrypoint: "astro-vue2/server.js",
  };
}

function getViteConfiguration() {
  return {
    optimizeDeps: {
      include: ["astro-vue2/client.js", "vue"],
      exclude: ["astro-vue2/server.js"],
    },
    plugins: [createVuePlugin()],
  };
}

export default function (): AstroIntegration {
  return {
    name: "astro-vue2",
    hooks: {
      "astro:config:setup": ({ addRenderer, updateConfig }) => {
        addRenderer(getRenderer());
        updateConfig({ vite: getViteConfiguration() });
      },
    },
  };
}

I have added vite-plugin-vue2 to Vite's configuration to allow Vite to build Vue2 components.

And the most remarkable thing is the getRenderer function. This function defines two entry points: serverEntrypoint and clientEntrypoint.

The serverEntrypoint is called at build time and SSR (Server Side Rendering). And the clientEntrypoint is called at hydration time.

Now let's delve into the details of these entry points.

server.js

The serverEntrypoint returns two functions, check and renderToStaticMarkup, but both tasks are very simple.

import Vue from "vue";
import { SSRLoadedRenderer } from "astro";
import { createRenderer } from "vue-server-renderer";
import { buildScopedSlots } from "./scoped-slots";

const check = async (
  Component,
  _props,
  _children
) => {
  return !!Component["staticRenderFns"];
};

const renderToStaticMarkup =
  async (Component, props, slotted, _metadata) => {
    const instance = new Vue({
      render: (h) =>
        h(Component, { props, scopedSlots: buildScopedSlots(h, slotted) }),
    });
    const html = await createRenderer().renderToString(instance);
    return { html };
  };

export default {
  check,
  renderToStaticMarkup,
};

The check function simply returns a boolean value indicating whether the given Component is the Vue2 component or not. so I used the staticRenderFns property, which only Vue2 has (although I don't know what the property is for...)

Also, The renderToStaticMarkup function should return a static HTML string based on the given Component. So I use the vue-server-renderer library to get the HTML from the Vue2 component.

client.js

The clientEntrypoint is simpler than the serverEntrypoint! Its role is to hydrate the Vue2 component based on the HTML created at SSR time.

import Vue from "vue";
import { buildScopedSlots } from "./scoped-slots";

Vue.config.ignoredElements.push("astro-slot");

export default (element: any) => {
  return async (
    Component: any,
    props: any,
    slotted: Record<string, string>,
    { client }: { client: string | null }
  ) => {
    new Vue({
      render: (h) =>
        h("astro-island", [
          h(Component, { props, scopedSlots: buildScopedSlots(h, slotted) }),
        ]),
    }).$mount(element, client !== "only");
  };
};

Note that Astro has the client:only option that allows Astro to behave SPA-like. So I need to pass a boolean as the second argument to $mount to tell Vue2 if it should hydrate the component.

Let's use the Vue2 integration!

All done! Now let's try out this integration! Before trying it, I added the Vue2 integration to Astro's configuration in advance. The astro-vue2 package is the same as main.js.

import { defineConfig } from "astro/config";
import vue2 from "astro-vue2";

// https://astro.build/config
export default defineConfig({
  integrations: [vue2()],
});

And I made a very simple Vue2 counter having a <slot />.

<template>
  <div>
    <button v-on:click="handleClick">Vue2 counter: {{ value }}</button>
    <p>slot: <slot /></p>
  </div>
</template>

<script>
export default {
  props: {
    defaultValue: {
      type: Number,
      required: true,
    },
  },
  data() {
    return {
      value: this.defaultValue,
    };
  },
  methods: {
    handleClick() {
      return this.value++;
    },
  },
};
</script>

This component is then used in an Astro component. Also, In this astro file, another Astro component is passed to the Vue2 component as a child component.

---
// @ts-ignore
import Counter from "../components/Counter.vue";

import Child from "../components/Child.astro";
import Layout from "../layouts/Layout.astro";
---

<Layout title="Astro and Vue2">
  <main>
    <Counter defaultValue={10} client:visible>
      <Child name={"Astro component"} />
    </Counter>
  </main>
</Layout>

Does this actually work properly...? Of course, it does!

I've created just three files, but all of Astro's functions work surprisingly fine. Also, you can build these components with the astro build command!

This code is on GitHub

All of the code used in the demo is available on GitHub. If you're interested, you can view the code in this repository.

ktmouk / astro-vue2

Astro with Vue 2.x

astro-vue2

This is the proof of concept if Astro can integrate with Vue 2, not 3. I got curious about whether Astro can support Vue2 technically and I made it. It isn't meant for production use.

How to run

It is easy to run this project, like below. If you don't install the pnpm, please install it referring to pnpm's installation in advaince.

# Run in dev mode.
pnpm install
pnpm dev

# Or build and look at it in preview mode. it also works fine!
pnpm install
pnpm preview

Folder structure

This repository consists of two packages managed by the pnpm workspace.

`/integration`

This is the library for Astro can integrate with Vue2. I made it while referencing Vue3 plugin, which is offical integration.

`/astro`

This is the Astro project to test above plugin. It uses plugins at astro.config.mjs.

License

MIT

View on GitHub

I hope this article will interest you in Astro! 🚀

How does React 18 work inside?

ktmouk — Sun, 22 Jan 2023 08:59:59 +0000

I recently read the React18 source code on GitHub and learned some exciting things. In this post, I want to describe how React18 works, by using figures.

I have also included GitHub links in some sections for readers who want to look at the code.

Fiber

First of all, I will explain what Fiber is. If you are interested in React, you may have heard the word "Fiber" somewhere. However, there are few documents and articles that explain Fiber.

What is Fiber?

In brief, one of the Fibers means one of the components, such as <MyComponent> and <div>. React builds a Fiber tree (it's like a DOM tree) for calculating where it has changed due to user interactions.

Besides, Fiber also means a unit of the task of the queue. React can pause the rendering per Fiber.

Why does React use the Fiber tree?

This is because React can more easily manage the data associated with a component, such as the priority, the component name, etc., than using a DOM tree.

What is the structure of the Fiber tree?

Let's see what the structure of the Fiber tree is, I have prepared a sample code below to describe it.

<html>
  <body>
    <script src="./node_modules/react/umd/react.development.js"></script>
    <script src="./node_modules/react-dom/umd/react-dom.development.js"></script>

    <div id="root"><!-- React will inject elements here --></div>

    <script>
      const App = () => {
        const child = React.createElement('span', null, 'world');
        return React.createElement('div', null, 'Hello', child);
      }
      const root = ReactDOM.createRoot(
        document.getElementById('root')
      );
      root.render(React.createElement(App, {}, null));
    </script>
  </body>
</html>

If you run the code, React will create the Fiber tree like this.

As you can see, This tree is almost the same as a DOM tree.
However, it has some differences from a DOM tree, like the one below.

The Fiber tree contains custom components such as <App>.
There are HostRoot and FiberRootNode components in the tree.

So, what are HostRoot and FiberNode?

FiberNode

FiberNode is the node indicated by the green circle in the figure. It has some properties named child, sibling, and return. These properties are for accessing its child, sibling, and parent.

FiberRootNode

FiberRootNode is the node indicated by the red circle in the figure. Its structure is entirely different from FiberNode, and it has a property called containerInfo that allows React to access the root element by accessing this property.

How does React create the Fiber tree?

Then, let's see how React creates the Fiber-tree.
I describe the rendering process using the sample code below and have added comments at key points in the code.

<html>
  <body>
    <script src="./node_modules/react/umd/react.development.js"></script>
    <script src="./node_modules/react-dom/umd/react-dom.development.js"></script>

    <div id="root"><!-- React will inject elements here --></div>

    <script>
      const App = () => {
        const child = React.createElement('span', null, 'world');
        return React.createElement('div', null, 'Hello', child);
      }
      // ★1 Create the FiberRootNode and the HostRoot
      const root = ReactDOM.createRoot(
        document.getElementById('root')
      );  

      // ★2 Create the ReactElement of `<App>`
      reactElement = React.createElement(App, {}, null)

      // ★3 Start render process
      root.render(reactElement);  
    </script>
  </body>
</html>

★1 Create the FiberRootNode and the HostRoot

When React calls the ReactDOM.createRoot method, it invokes the function named createFiberRoot to create the FiberRootNode and the HostRoot.

You can look at the code on GitHub.

Creating FiberRootNode

Creating HostRoot

After creating the FiberRootNode and the HostRoot, the Fiber tree looks like this.

★2 Create the ReactElement of `<App>`

Then, React creates the ReactElement, which is required for calling the root.render function.

Note that this object structure is completely different from Fiber. If you use the JSX format, you typically don't need to use the React.createElement method because a compiler like Webpack will convert JSX to the React.createElement at compile time. (docs)

★3 Start rendering

Finally, the root.render function is called, and React starts rendering. This process has three steps.

React recursively converts ReactElements to Fibers and adds them to the Fiber tree until it reaches the terminal node.
When React reaches the terminal node, it returns to the parent while saving a DOM element which is made from a Fiber to the property named stateNode.
In the end, when React reaches the root, it appends the completed DOM elements to div#root with appendChild.

The explanation is complicated, so I made a GIF image to make it easier to understand.

The stateNode property generally holds the DOM element, including itself and its children, but functional components (such as <App>) don't have stateNode. Their stateNode property is always null.

Finally, the function finds the closest stateNode (except for null) from the root and appends it to the div#root. Then, a user can see the message "Helloworld" on the screen.

Rendering phase and Commit phase

I explained the flow of the rendering process in the previous section. In this section, I will explain the rendering process in more detail.

Two phases in the rendering process

The rendering process can be divided into two phases, the rendering phase, and the commit phase.

In the rendering phase, React converts ReactElements to DOM elements and stores them in stateNode. Then, in the commit phase, React appends the DOM elements to div#root.

It means React doesn't change DOM elements during the rendering phase, and any changes are never shown on the screen until the commit phase starts. So, React can pause or resume the rendering phase anytime if it takes longer.

This behavior provides a good user experience because React doesn't block the JavaScript thread for long. JavaScript is a single-threaded language, and if React blocks the thread for too long, the user may think the browser has frozen up.

On the other hand, in the commit phase, React can block the JavaScript thread. This is not good for the user experience. However, React just updates DOM elements based on the Fiber tree in the commit phase, so React completes its tasks quickly and releases the thread.

In what cases does React pause the rendering phase?

I was curious about this and looked into cases of the rendering phase can be paused. According to this issue below, React can only pause the rendering phase only if we use startTransition or Suspense.

Bug: time slice not work in react 18 #24392

yuwanlin posted on Apr 18, 2022

in react@16.8.0, a long task will be sliced multi short task, demo: https://stackblitz.com/edit/react-ts-aqwejz

but in react@18.0.0, it will be only a long task, demo: https://stackblitz.com/edit/react-ts-ezgtzn

Is it a react18 time slice bug or feature?

React version: 16.8.0 & 18.0.0

Steps To Reproduce

run a example app like below
open inspector -> performance, then record and analyze

Link to code example: 16.8.0: https://stackblitz.com/edit/react-ts-aqwejz 18.0.0: https://stackblitz.com/edit/react-ts-ezgtzn

The current behavior

in react@18.0.0, long task not be sliced

The expected behavior

in react@18.0.0, long task will be sliced

View on GitHub

So if you want to allow React to pause the render phases, you need to wrap your code with startTransition or Suspense, like this.

React.startTransition(() => {
  root.render(React.createElement(App, {}, null));
})

Then, How does React start the rendering phase inside?
React calls the workLoopConcurrent or workLoopSync function to start the rendering phase. Both functions start the rendering phase, but workLoopConcurrent can pause it, and workLoopSync cannot.

The workLoopConcurrent function calls the shouldYield function to check whether React should pause the rendering phase. The shouldYield function returns true if more than 5ms has passed since the phase start.

You can look at the code on GitHub.

workLoopConcurrent

workLoopSync

How does React decide which function to call? It depends on Lane.

What is Lane?

Briefly, Lane is a 32-bit mask flag. Each bit represents a priority and a type of task. The closer the lane is to '0', the higher the priority.

Also, the NoLane is a special one. Basically, this Lane is for a default value.

You can take a look at the list of lanes below.
ReactFiberLane.old.js (GitHub)

As you can see, the lane is just an integer. React can use bitwise operators to create a bitmask containing some lanes or check if the bitmask has a particular lane.

const SyncLane = 0b0000000000000000000000000000001;
const InputContinuousLane = 0b0000000000000000000000000000100;

// Merge lanes by using `|` operator
// Returns a bitmask containing `SyncLane` and `InputContinuousLane`.
const merged = SyncLane | InputContinuousLane; 
console.log(merged); // 0b0000000000000000000000000000101

// Check if the bitmask have `InputContinuousLane`
console.log((merged & InputContinuousLane) != 0); // true

In the rendering process, React has to handle a lot of flags. The bitmask is a good solution for managing flags, compared to having them with individual variables.

How does React choose lanes?

There are several ways that React can select a lane, but one of the simplest is to select a lane based on an event type, such as a click event.

Let's look at the sample code below. When you click the button, the text in the button changes from "Click" to "Clicked".

const App = () => {
  const [test, setTest] = React.useState('Click')

  const onClick = () => {
    setTest('Clicked')
  }

  return React.createElement('button', { onClick }, test);
}

const root = ReactDOM.createRoot(document.getElementById('root'));
root.render(React.createElement(App, {}, null));

After the page loaded, React calls the listenToAllSupportedEvents function to add many event listeners, allowing React to catch almost all user events. You can look at these listeners in the DevTools.

When you click a button, the above event listener is fired before the onClick callback is triggered, and React decides the lane based on an event type. In the case of the sample code, React selects the SyncLane when a click event occurs. It means that the click event is a high priority.

This is because a user generally wants the browser to respond quickly when the user clicks something. On the other hand, a mouse-move event is not higher than a click event.

You can look at the code on GitHub.

listenToAllSupportedEvents

I looked at the usage of lanes in the source code and made a list. I put "unknown" in the place where I couldn't find the usage.

Name	Usage
NoLane	Default value
SyncLane	ClickEvent, MouseMoveEvent, etc.
InputContinuousHydrationLane	Unknown. It is probably used for Suspense and hydrateroot
InputContinuousLane	MouseEnter, etc.
DefaultHydrationLane	Unknown. It is probably used for Suspense and hydrateroot
DefaultLane	It is mainly used for initial rendering.
TransitionHydrationLane	Unknown. It is probably used for Suspense and hydrateroot
TransitionLane1~16	It is used for `startTransition`. React uses a different number each time, and if React previously used TransitionLane16, it will use TransitionLane1.
RetryLane1~5	React uses it when `Suspense` is still loading.
IdleHydrationLane	Unknown. It is probably used for Suspense and hydrateroot
IdleLane	Unknown
OffscreenLane	Unknown. It's probably used for the offscreen feature, which is not implemented yet.

How does React use lanes?

So how does React use lanes? It is not easy to list all the uses, because React uses them in many different ways, but the main uses are as follows.

React uses the lanes contained in each Fiber to determine if a Fiber has any updates and should be re-rendered.
React uses a lane to decide if it can pause the rendering phase. (This is what I described in the previous section.)

About the scheduling

In the previous section, I described the rendering phase and the lane. In this section, I will explain the scheduling that React has.

React has a scheduling system. The scheduler queues the rendering phase as one of the tasks and invokes it with a delay. The scheduler queues the rendering phase when the initial load or any event occurs, such as a click event.

And interestingly, SyncLine (used for click events, etc.) is special because React chooses a scheduler system based on whether the current lane is SyncLine or not.

Let's see what the difference is between SyncLine and other lanes.

Case 1. non-SyncLane

1. Create and queue a task.

After any events or an initial page load happens, React creates a task object like the one below.

var newTask = {
  id: // The task id. It is used to sort tasks.
  callback: // This property holds the function called when the task is dequeued, such as the function to start the rendering process.
  priorityLevel: // Unknown.
  startTime: // The time this task was performed.
  expirationTime: // The expiration time. The higher the lane priority, the shorter it is. When it expires, React executes it without pausing.
  sortIndex: // The index for sorting tasks. Basically, its value is the same as startTime. If different tasks have the same sortIndex, React will use `id` instead.
};

Then, React pushes it into the array for managing tasks. This array consists of a binary heap.

As a feature of the binary heap, the head of the array is always a high-priority task. React uses the sortIndex property as a priority, and sorts tasks based on the sortIndex.

A binary heap is an ideal structure for managing tasks because React can pick the highest priority task by simply accessing the head of an array.

2. Add the `flushWork` function to the macrotask queue.

After adding the task to a binary heap, React doesn't immediately execute it. React executes the task via the flushWork function. This function is called through the macrotask queue.

Note that the macrotask queue is not the concept of React. this queue is implemented by JavaScript. JavaScript has two queues, MacroTask and MicroTask. Both have some sort of queue structure, but these roles are completely different.

MacroTask is used to execute an event callback, such as setTimeout mousemove, and MicroTask is used to execute Promise callback. For more information, see javaScript.info.

React uses MacroTask and MicroTask depending on whether the current lane is SyncLine or not. If the current lane is SyncLane, React uses MicroTask to perform the flushWork function. Otherwise, React will use MacroTask instead.

This is because JavaScript generally executes the MicroTask before the MacroTask, and SyncLane is used for an important event like a click event, so React needs to complete the SyncLane task as quickly as possible for user experience.

In this section, we are looking at the case of non-SyncLane, so React uses the MacroTask.

3. The `flushWork` function is called via EventLoop.

Then, the flushWork function is called via EventLoop. In the flushWork function, React picks the highest task from the binary heap and executes it (in many cases, this task is to start the rendering phase).

The above process is summarized in the figure below.

Case 2. SyncLane

1. Add the task to the array.

If the current lane is SyncLane, React adds the task to the array named syncQueue. This array is not the binary heap structure and doesn't need to be prioritized. This is because this array is only used for SyncLane tasks, not for tasks from other lanes.

2. Add the `flushSyncCallbacks` function to the MicroTask queue.

Then, React queues the function named flushSyncCallbacks to the MicroTask queue, not the MacroTask queue. This function will execute all of the tasks in the syncQueue array in a row.

3. The `flushSyncCallbacks` function is called via EventLoop.

Then, the flushSyncCallbacks function is called via EventLoop, and all of the tasks in the syncQueue array are executed.

The above flow is shown in the figure below.

How does React updates the components?

In the previous sections, I have explained the rendering phase, commit phases, lanes, and scheduling. In this section, I will describe how React updates the components.

I use the following sample code to describe the update flow. This sample code has hooks because it is required to fire any events.

When you hover over the button, a browser will change the text from 'Helloworld' to 'ABCworld'.

<html>
  <body>
    <script src="./node_modules/react/umd/react.development.js"></script>
    <script src="./node_modules/react-dom/umd/react-dom.development.js"></script>

    <div id="root"><!-- React will inject elements here --></div>

    <script>
      const App = () => {
        const [test, setTest] = React.useState('')

        const onMouseMove = () => {
          setTest((test) => 'A')
          setTest((test) => test + 'B')
          setTest((test) => test + 'C')
        }

        const child = (test === '') ?
          React.createElement('span', {}, 'Hello') :
          React.createElement('b', {}, test)

        return React.createElement('div', { onMouseMove },
          child,
          React.createElement('span', {}, 'world')
        );
      }

      const root = ReactDOM.createRoot(document.getElementById('root'));
      root.render(React.createElement(App, {}, null));
    </script>
  </body>
</html>

Let's look at the update flow after the initial render. The following fiber tree has already been created in the initial render.

What happens when you move the mouse over the <div>?

1. Create a circular list of hooks.

When you hover the <div>, the onMouseMove function in the sample code is called.

const onMouseMove = () => {
  setTest((test) => 'A')
  setTest((test) => test + 'B')
  setTest((test) => test + 'C')
}

The setTest function is the return value of the useState function and is used to update the state. So let's see what the setTest function does inside.

When setTest function is called, it calls the dispatchSetState function, and then, it creates the object named update.

var update = {
  lane: lane,  // This property holds the lane. In the case of the sample code, the lane is `InputContinuousLane'.
  action: action, // This property holds the `onMouseMove` function.
  hasEagerState: false, // It is used to improve performance.
  eagerState: null, // It is used to improve performance.
  next: null // The property has a reference to the next `update` object.
};

These update objects consist of a circular list. React adds an update object to the end of the circular list each time setTest is called. In the example, React eventually makes the circular list look like this.

Then, React sets this circular list to the memoizedState of the Fiber that owns this hook. Also, React doesn't calculate the result (ABC) at this point.

2. The rendering phase is called via EventLoop.

Then, as the previous section described, React pushes the task to the MacroTask queue to start the render phase. After that, EventLoop invokes the task and starts the render phase.

3. Copy and reuse the previous Fiber tree.

The difference from the initial rendering is that React already has a Fiber tree. React reuses the previous tree as much as possible instead of creating a new one.

So, where does React have to change? In the example code case, React has to change the <div><span>Hello</span><span>world</span></div> element to <div><b>ABC</b><span>world</span></div>.
It means, React needs to do the following:

Remove the <span>Hello</span> element.
Add the <b>ABC</b> element.

In the React world, we call this process Reconciliation.

However, React cannot change the real DOM at the render phase because the render phase could be paused for some reason. If React changes the DOM and the render phase is paused, the user will see the website which may be incomplete.

So, React stores the data in the Fiber instead, and updates the DOM during the commit phase.

This is the time to use a technique called "Double Buffering". React copies the previous Fiber tree instead of creating it from the scratch. In the figure below, the Fiber tree that is currently being worked on is labeled "WIP (work in progress)", and the Fiber tree that is currently displayed is labeled as "Current".

Note that JavaScript's "copy" basically means "shallow copy". React copies a Fiber, but not its property. The properties of Fiber refer to the same value as the original Fiber. For example, in the figure below, stateNode and memoizedState are relevant (The memoizedState is omitted due to space limitations.)

Also, React cannot update the stateNodes (DOM) at this point because all stateNodes are already appended to div#root via the root element and any update of stateNodes will be rendered immediately on the screen.

4. Appends new Fibers

Then, React executes the circular list that is in memoizedState property, and gains a new state, ABC.
React then creates a new Fiber that has the <b>ABC</B> element, and appends it to the WIP tree.

And React can create the stateNode for the new Fiber at this point. This is not a problem because this Fiber is new and is not appended to div#root, so a browser will not render this stateNode.

5. Copy unchanged Fibers

As same as 3. step, React copies the unchanged Fibers. In the below figure, React copies the <span>world</span> element to WIP.

6. Mark the Fiber to be deleted.

React marks Fibers that should be deleted during the commit phase. In this example, it's <span>Hello</span>. React stores a reference to this fiber in a parent property named deletion, an array.

This is the end of the render phase. In the next section, we will look at the commit phase.

7. Update and delete DOM elements in the commit phase

In the commit phase, React updates and deletes elements in the DOM tree based on the Fiber tree and the deletion array. First, React deletes the <span>Hello</span> element with removeChild from stateNode　(written in red in the figure) of the parent Fiber.

Then, React inserts a new DOM element, <b>ABC</b>, into the stateNode of the parent Fiber by using insertBefore.

The user will then see the new text "ABCworld" on the screen.

8. Makes the WIP tree the current tree.

For the next update, React sets the WIP tree to the current property of the FiberRootNode as the current tree. The old "current tree" will no longer be used.

That's all. The rendering is now complete.

Conclusion

React works with complex algorithms and structures and these are hard to understand. but, according to principles, React prioritizes performance and a good developer experience instead of being elegant. So I think this is why most developers love React and prefer to use it.

DEV Community: ktmouk

9 Tips for Building a Secure Web Application with tRPC, Next.js, Prisma, Turbo, and NextAuth

Directory Structure

prisma/

schema/

services/

trpc/

Validate Parameters in Each Package

Setting Custom tRPC Error Formatter

Manage ENVs with T3 Env

Disable fetchCache If You Don't Use It

Add The CSP Header

Add CSRF prevention

Add the Cache-Control header

Use server-only

Test Backend Code

Conclusion

ktmouk / minute

⏰ The open-source time tracking app for individuals.

minute

About

Demo

Screenshots

Features

Folders

Categories

FactoryJS - the TypeScript version of factory_bot

What is factory_bot?

What about the TypeScript world?

Requirements

Is it really possible?

API design

props

vars

create

Examples

Is vars really required?

How does it work inside?

Other features

trait

dependent props / vars

Prisma Plugin

Conclusion

factory-js / factory-js

🏭 The object generator for testing.

FactoryJS

Features

Quick Start

Using Vue 2 component in Astro, not Vue3 .... is it possible?

Introduction

Vue2 will reach EOL at the end of 2023

Let's make Vue2 integration myself!

main.js

server.js

client.js

Let's use the Vue2 integration!

This code is on GitHub

ktmouk / astro-vue2

Astro with Vue 2.x

astro-vue2

How to run

Folder structure

/integration

/astro

License

How does React 18 work inside?

Fiber

What is Fiber?

Why does React use the Fiber tree?

What is the structure of the Fiber tree?

FiberNode

FiberRootNode

How does React create the Fiber tree?

★1 Create the FiberRootNode and the HostRoot

★2 Create the ReactElement of <App>

★3 Start rendering

Rendering phase and Commit phase

Two phases in the rendering process

In what cases does React pause the rendering phase?

Bug: time slice not work in react 18 #24392

`prisma/`

`schema/`

`services/`

`trpc/`

`/integration`

`/astro`

★2 Create the ReactElement of `<App>`

2. Add the `flushWork` function to the macrotask queue.

3. The `flushWork` function is called via EventLoop.

2. Add the `flushSyncCallbacks` function to the MicroTask queue.

3. The `flushSyncCallbacks` function is called via EventLoop.