DEV Community: chimame

Cooperation between Cloudflare Workers has become amazing thanks to RPC support

chimame — Tue, 09 Apr 2024 16:43:35 +0000

On April 5, 2024, Japan time, Cloudflare announced communication between Cloudflare Workers using RPC.

https://blog.cloudflare.com/javascript-native-rpc

I think it's no exaggeration to say that this has solved several issues and at the same time raised the convenience of building applications on Cloudflare not by one level, but by more than two levels.
This RPC support has made Service Bindings even more user-friendly, so I would like to introduce it.

The completed code is here, so if you don't have time, please refer to it.

https://github.com/chimame/connect-remix-and-prisma-d1-using-rpc-on-cloudflare-pages

Prerequisites

Bundling Remix and Prisma will exceed 1MB in size. Therefore, deployment will not complete successfully with a free Cloudflare account. I will explain this on the premise of solving the problem.
Of course, the main topic is the content of Service Bindings using RPC announced by Cloudflare.

https://github.com/chimame/remix-prisma-d1-on-cloudflare-pages

Service Bindings

Service Bindings allow one Cloudflare Worker to call another Cloudflare Worker.

The point is that you don't have all the workers in one place, you divide them up and place them so that you can call workers from one worker to another. There are also some benefits to partitioning.

Cloudflare Workers file size can be kept small
Cache each Cloudflare Workers process

And so on. Of course, there are benefits to keeping it small, but if you divide it too much, there are also disadvantages similar to microservices, so be careful about that.
However, Cloudflare Workers also has size limitations (depending on your subscription plan). If it were free, there would be a limit of 1MB in post-build (compressed) size. Serivce Bindings are also a very useful feature to avoid size constraints.

However, until now, there was a problem of what to do with the interface between the two workers if they were simply split using Serivce Bindings. This problem has been solved by this RPC, so I will explain how to use it and divide it into parts.

Create a monorepo with 2 Workers

Currently, the size of Wasm in Prisma is a little large, so if you bundle Remix and Prisma, it will exceed 1MB. So, unfortunately, those who wanted to run the free version had to use other methods.
However, it has become possible to smartly avoid this using the RPC that was announced this time, so I will write about it.

This time, I will write an outline of the composition of the completed form below.

Split into two Workers as above.

Workers specialized in DB access using Prisma
Workers who render with received data using Remix

In the case of Remix, it uses Cloudflare Pages, so it is called Cloudflare Pages Functions, but in reality it is Cloudflare Workers, so there is little difference. So, to avoid the size limit, build the application by dividing it into two workers as shown above.

Since we will be creating two Workers, we will create the project. yarn I will describe it with a monorepo configuration using, but pnpm you can create it with whatever you like.

We will arrange it like this.



├── README.md
├── package.json
├── yarn.lock
└── pakcages/
    ├── prisma(workers)
    └── remix(workers)

Initialize Remix

Place the Remix as shown below without thinking about anything in particular.



mkdir -p packages/remix
cd packages/remix
npx create-remix@latest --template remix-run/remix/templates/cloudflare

Initial setup of Prisma

Last time I put it in the same package.json as Remix, but this time I'll separate it. There is no problem if you configure it from the project root using the following command.



mkdir -p packages/prisma
# Create packages/prisma/package.json with only `name` written
yarn workspace <prisma package name> add -D prisma
yarn workspace <prisma package name> add @prisma/client @prisma/adapter-d1
yarn workspace <prisma package name> run prisma init --datasource-provider sqlite

schema.prisma Add the following description to the last created file to complete it.



// scheme.prisma
generator client {
 provider = "prisma-client-js"
 previewFeatures = ["driverAdapters"]  // <-- ADD
}

datasource db {
 provider = "sqlite"
 url      = env("DATABASE_URL")
}

The initial settings for Workers on the Prisma side are now complete.

Database migration

Prisma's D1 support is still in Preview stage. Therefore, migration to D1 is not supported. The Prisma introductory article introduces the procedure for migrating from standard output to D1, but it is not suitable for actual operation. Therefore, if you want to use it, it would be a good idea to move Prisma's migration DDL to the D1 migration file as supported in the commit below.

https://github.com/chimame/connect-remix-and-prisma-d1-using-rpc-on-cloudflare-pages/commit/4718ddbeda21db2e5496dd7558d736bdd1a974f0



#!/usr/bin/env zx

await $`mkdir -p ./migrations`

const packages = await glob(['prisma/migrations/*/migration.sql'])
for (let i = 0; i < packages.length; i++) {
  const migrationName = packages[i]
    .replace('prisma/migrations/', '')
    .split('/')[0]
  if (!fs.existsSync(`migrations/${migrationName}.sql`)) {
    await $`cp ${packages[i]} migrations/${migrationName}.sql`
  }
}

await $`yarn run wrangler d1 migrations apply prisma-rpc-db --local`

Process creation for Prisma Workers

First, let's write the process to access the Datebase.

Initial settings for Workers

The Prisma side package only includes Prisma, so first we will add the settings for Workers to run.



yarn workspace <prisma package name> add -D wrangler @cloudflare/workers-types
yarn workspace <prisma package name> run wrangler create D1 <D1 database name>

wrangler.toml I will describe the settings of the D1 I created . Below is an example.



# packages/prisma/wrangler.toml
name = "prisma-d1-rpc-sample"
main = "src/index.ts"
compatibility_date = "2022-04-05"
compatibility_flags = ["nodejs_compat"]

[[d1_databases]]
binding = "DB" # i.e. available in your Worker on env.DB
database_name = "prisma-rpc-db"
database_id = "__YOUR_D1_DATABASE_ID__"

Creating Workers using Prisma

Next is the actual code for the Workers on the Prisma side, and I will write the code like this.



// pakcages/prisma/database/client.ts
import { PrismaClient } from "@prisma/client";
import { PrismaD1 } from "@prisma/adapter-d1";

export const connection = (db: D1Database) => {
  const adapter = new PrismaD1(db);
  return new PrismaClient({ adapter });
}



// packages/prisma/src/index.ts
import { WorkerEntrypoint } from "cloudflare:workers";
import { connection } from "./database/client";

export interface Env {
  DB: D1Database,
}

export class UserService extends WorkerEntrypoint<Env> {
  fetchUsers() {
    const db = connection(this.env.DB)
    return db.user.findMany();
  }

  async createUser(data: { name: string, email: string }) {
    const db = connection(this.env.DB)
    const user = await db.user.create({
      data,
    });

    return user;
  }
}

export default {
  // An error occurs when deploying this worker without register event handlers as default export.
  async fetch() {
    return new Response("Healthy!");
  },
};

client.ts I don't think there is any need to explain anything, but it simply defines a function to connect to D1 using Prisma. src/index.ts The side is the code of the Workers main body. The target of Service Bindings is exporting UserService. Although it is defined using named export, there is no problem with default export. UserServicehas two functions and has defined two functions: one that returns user data fetchUsersand one that creates a user . createUserThis function uses Prisma to access D1 and perform SELECT and INSERT processing.
I don't think it's particularly difficult if you just look at it. The important thing here is WorkerEntrypoint<Env>. In particular Env, it makes D1 settings this.env.DBaccessible by importing them, and serves as the basis for defining the RPC-compatible Service Bindings. This allows processing of D1 and Prisma to be confined to the Workers here.

Creating a Remix (Pages Functions)

Configuring Service Bindings

Remix has most of the settings in the initial setup, but this time we will be using Service Bindings using RPC, so we will need the following wrangler.toml settings Env.



# packages/remix/wrangler.toml
name = "remix-rpc-sample"

[[services]]
binding = "USER_SERVICE"
service = "prisma-d1-rpc-sample"
entrypoint = "UserService"



// packages/remix/load-context.ts
import { type PlatformProxy } from "wrangler";
import type { UserService } from "@rpc-sample/prisma/src";

interface Env {
  USER_SERVICE: Service<UserService>; // <---ADD
}

type Cloudflare = Omit<PlatformProxy<Env>, "dispose">;

declare module "@remix-run/cloudflare" {
  interface AppLoadContext {
    cloudflare: Cloudflare;
  }
}

First, settings are written in wrangler.toml so that Prisma's Workers can be referenced from the Remix side. By writing this, you will be able to use Service Bindigs settings locally. entrypointThis setting is specified here , but this setting is required when writing Workers to be bound by named export. Conversely, this is not necessary for default export. The content of
load-context.ts what you write Env is wrangler.toml defined as a type.
By having these two things in place, the true power of Service Bindings using RPC, which will be described later, will be demonstrated.

There is a point to note, although wrangler types a Remix npm script is defined, it does not output the type shown wrangler types above even if you use it Service<UserService> . I don't think it supports RPC yet, so please correct it manually.

Load DB data via Service Bindings

In Remix, loader data is loaded using , but the program that loads data using the configured Service Bindings is as follows.



// packages/remix/app/routes/users/handlers/loader.ts
import type { LoaderFunctionArgs } from "@remix-run/cloudflare";

export const loader = async ({ context }: LoaderFunctionArgs) => {
  using users = await context.cloudflare.env.USER_SERVICE.fetchUsers();

  return { users }
}

The actual data loading process is on the Workers side of Prisma, so Remix just calls it from the connection settings. But this is TypeScript. usersWhat happens to this variable?

The type is defined like this. This is the greatness of Service Bindings that support RPC. Since type information is linked in this way, you can greatly benefit from TypeScript. This tRPC means that you can implement pretty much the same thing within Cloudflare as you did a while ago .

useLoaderData Since we have a type , all we have to do is get the data from this as usual and render it.



// packages/remix/app/routes/users/route.tsx
import { Form, useLoaderData } from "@remix-run/react";
import { loader, action } from "./handlers";

export default function UserPage() {
  const { users } = useLoaderData<typeof loader>()

  return (
    <div>
      <Form method="post">
        <label htmlFor="name">Name</label>
        <input type="text" id="name" name="name" />
        <br />
        <label htmlFor="email">Email</label>
        <input type="email" id="email" name="email" required />
        <button type="submit">Add</button>
      </Form>
      <h1>Users</h1>
      <ul>
        {users.map((user) => (
          <li key={user.id}>{user.name}</li>
        ))}
      </ul>
    </div>
  )
}

export { loader, action };

Launching the application

Service Bindings using this RPC could not be started using Vite ( getPlatformProxy ). It seems that it is not supported yet. Therefore, we need to run it with wrangler pages dev, which is the actual start script, which is another startup command.

Prisma's Workers will also start, so concurrentlystart them at the same time.



{
  ...
  "scripts": {
    "dev": "concurrently \"yarn run dev:prisma\" \"yarn run dev:remix:build && yarn run dev:remix:start\"",
    "dev:prisma": "yarn workspace @rpc-sample/prisma dev",
    "dev:remix:build": "yarn workspace @rpc-sample/remix build",
    "dev:remix:start": "yarn workspace @rpc-sample/remix start"
  },
  ...
}

Now, /users when you access the Remix screen and access the path, you should see a screen displaying user registration and the registered user.

The RPC announced this time makes it possible for Remix and Prisma to work together so that they can share type information while separating each worker. I tried it with a free version account and have confirmed that it can be successfully deployed and works.

`using` What?

loader During the Remix process using, you will notice that it is smooth. This appeared in TypeScript 5.2, but this is a specification in Stage 3 of ECMAScript.

https://github.com/tc39/proposal-explicit-resource-management

If you write very roughly, it will free up resources in a good way. (Seriously, I'm writing this in a rough way, so please try to understand it yourself) So why are you using it this time? That's explained in detail in this document.

https://developers.cloudflare.com/workers/runtime-apis/rpc/lifecycle/

It's too long to read it all, so I'll summarize it:

using is a Stage 3 specification, but it will also be incorporated into V8 soon.
The return value of Cloudflare Workers' RPC is held in memory on the server side (Prisma side Workers in this article).
To explicitly release memory on the server side, Symbol.dispose call the so-called disposer from the client side (Remix side in this article) and instruct the server side to release it.
If you want to keep the variable on the client side, dup copy it.

That's it. That's right. The value returned from this RPC is not held in the caller's memory space, but on the server side. (The person who implemented it is amazing) is done using when freeing memory. Therefore, this specification is also used Symbol.dispose to release memory on the server side.
In other words, Service Bindings are not exchanging data using a protocol like http, but rather the Workers connected using Service Bindings are running on some other host. It may be close to something like that.

Finally

Announcing RPC greatly improves the usability of Cloudflare Workers' Serivce Bindings. It is far beyond imagination that so much can be implemented using just the smart writing style of the class syntax. It should now be much easier to implement the front end and back end with just Cloudflare Workers.
Please try using it when you come up with a design that requires collaboration between Cloudflare Workers.

Understanding how to make Cloudflare Pages compatible with Vite, which became stable with Remix 2.7

chimame — Fri, 01 Mar 2024 02:33:35 +0000

Remix 2.7 has been released. From 2.7 onwards, Vite compatibility, which has been unstable until now, has been adopted as the official version.
Prior to 2.7, the Node.js runtime provided something that worked, but not something that worked with Cloudflare Pages. However, at the same time as the release of 2.7, something that works with Cloudfalre Pages was released, so I will summarize the results of my research on what has changed and how it is being handled.

https://github.com/remix-run/remix/blob/main/CHANGELOG.md#v270

Several bugs were fixed after the release of 2.7.0, so we recommend using the bug-fixed version of the migration instead of 2.7.0.
Also, there is an official document for migrating to the Vite version, so I think reading it will help you understand better.

https://remix.run/docs/en/main/future/vite#migrating

Differences in initial `package.json`

First, let's compare the differences in the initial packages between the conventional version and the Vite version.

Differences in installation commands

Please note that the templates are separate, so the traditional version and the Vite version are different.

npx create-remix@latest --template remix-run/remix/templates/cloudflare-pages

npx create-remix@latest --template remix-run/remix/templates/vite-cloudflare

Difference between npm script and npm packages

The package.json actually generated from the above command differs as follows.

Of course, Vite etc. are added to the installed packages. You can see that the packages are different, but you can also see that the npm scripts are also different.

Added npm scripts for deploy postinstall and typegen
Add dev and change start

There have been many changes, and I will explain each of the changes.

What are `typegen` and `postinstall`?

First, I will explain the additional npm scripts typegen and postinstall.
postinstall allows you to define commands that are automatically started after executing the npm install command. Therefore, in the Vite version, the script defined in postinstall will be executed by executing the command equivalent to npm install. The important content of postinstall is simply reading the typegen npm script. So the main body is a typegen npm script.

The true identity of `typegen`

I'm hitting a command called wrangler types which is what typegen is doing. You can understand what this is doing by reading the Wrangler documentation.

https://developers.cloudflare.com/workers/wrangler/commands/#types

In short, it generates worker-configuration.d.ts, the type file required for the application, from the wrangler.toml file used in Cloudflare's development environment and deployment.

Specifically, if you have a wrangler.toml file like the following

vars = { API_HOST = "example.com" }

kv_namespaces = [
  { id = "MY_KV", binding="MY_KV" }
]

d1_databases = [
  { binding = "DB", database_name = "TestDB", database_id = "test-db-id" }
]

interface Env {
    MY_KV: KVNamespace;
    API_HOST: "example.com";
    DB: D1Database;
}

This worker-configuration.d.ts type file is very important as it will be used later in Remix. This is because this file corresponds to the previous version, remix.env.d.ts. The Vite version does not use remix.env.d.ts, so be sure to generate it as written in the migrate documentation.

Notes on type generation

It's convenient because it automatically generates a type file, but there are some things to keep in mind.
What it is is the way wrangler.toml is written. In the previous example, there are two ways to write vars, and I wrote one of them, but the other is written as follows.

# Changed vars = { API_HOST = "example.com" }
[vars]
API_HOST = "example.com"

kv_namespaces = [
  { id = "MY_KV", binding="MY_KV" }
]

d1_databases = [
  { binding = "DB", database_name = "TestDB", database_id = "test-db-id" }
]

This is also a correct description, but if you run typegen's npm script in this state, worker-configuration.d.ts like the following will be created.

interface Env {
    API_HOST: "example.com";
    kv_namespaces: [{"id":"MY_KV","binding":"MY_KV"}];
    d1_databases: [{"binding":"DB","database_name":"TestDB","database_id":"test-db-id"}];
}

This is not the type file format that Remix is aiming for, so be careful when writing it.

`wrangler.toml` trap in Cloudflare Pages

A very big trap lurks. The README of Cloudflare Pages, which is compatible with the Vite version of Remix, has the following description.

> [!WARNING]  
> Cloudflare does _not_ use `wrangler.toml` to configure deployment bindings.
> You **MUST** [configure deployment bindings manually in the Cloudflare dashboard][bindings].

Since it generates the form, wouldn't it also be reflected during deployment? For some reason, wrangler pages deploy does not reflect the contents of wrangler.toml. So, I'm telling you to first configure it on the Cloudflare console.

Two ways to launch a development environment

In the Vite version of 2.7, there are two methods. That's the modified dev and start npm script. The previous version of dev's npm script is written as follows.

  "scripts": {
    ...
    "dev": "remix dev --manual -c \"npm run start\"",
    ...
    "start": "wrangler pages dev --compatibility-date=2023-06-21 ./public",
    ...
  },

So, if you start dev, it will start up to start. (The conventional version is also built using remix dev.) So, what's going on with the Vite version?

  "scripts": {
    ...
    "dev": "remix vite:dev",
    ...
    "start": "wrangler pages dev ./build/client",
    ...
  },

dev and start are not linked. Works as separate commands. Regarding dev, some commands that work with Vite are defined. As for start, the command to start with wrangler is defined in the built file as before.
So what has changed here is that start starts the development environment using wrangler, which is almost the same as the previous version, and dev is the command to start the development environment with the newly added Vite.

The reality of `remix vite:dev`

Let's find out what the newly added remix vite:dev is. The explanation will be long if you follow from the cli command, so I will only write the main points.
Since it is the Vite version, when you look at the Vite configuration vite.config.ts, you will see the following description for Cloudflare called cloudflareDevProxyVitePlugin.

import {
  vitePlugin as remix,
  cloudflareDevProxyVitePlugin as remixCloudflareDevProxy,
} from "@remix-run/dev";
import { defineConfig } from "vite";
import tsconfigPaths from "vite-tsconfig-paths";

export default defineConfig({
  plugins: [
    remixCloudflareDevProxy(),
    remix(),
    tsconfigPaths()
  ],
});

In this guy's code, we get to the reality of Vite Server, a so-called development environment.

https://github.com/remix-run/remix/blob/df0a668d416014f19313419dc7701ddcbe4ee312/packages/remix-dev/vite/cloudflare-proxy-plugin.ts#L59-L85

In this configureServer, we call the wranlger API called getPlatformProxy. It is no exaggeration to say that this is the reality of remix vite:dev.

https://developers.cloudflare.com/workers/wrangler/api/#getplatformproxy

Simply put, wrangler acts as a proxy, and Vite Server actually runs. So, why does wrangler need to act as a proxy? What is returned from this getPlatformProxy is that running Cloudflare's own services (such as D1, R2, KV, etc.) on the Vite Server side returns the API. That is a major feature. I wrote a program to access D1 in the Remix code in the development environment, and it works on Vite Server because Remix on Vite Server runs the Cloudflare environment API that was proxied with getPlatformProxy.

Notes on `remix vite:dev`

If you have read this far and have the following questions, you are in good hands.

I want to check the operation in a workerd environment.

that's right. It's still the same as wrangler providing Cloudflare's API and running on Vite Server, that is, Node server. Therefore, since it is not a wranglerd environment, code that depends on Node's API will run without doing anything.
I think that's one of the reasons why the start npm script remains in order to finally check the operation. On the start side, we will use wrangler to raise the development environment, so we will check the operation in the workerd environment.

“Then why don’t we just say start?”

I certainly think so. However, there are other problems with start, and you need to be careful if you use it regularly.

https://github.com/remix-run/remix/issues/7466

Since the Vite version has just been released, there are some issues, but efforts to improve this have already begun, so I think it would be best to wait patiently, or since it is OSS, it would be better to cooperate.

Build process up to deployment in Vite version

The method of starting the development environment has certainly changed with Vite, but the main focus is building. Since it is a Vite version, of course the build will be done with Vite. By the way, the conventional version is built using esbuild.

The reality of `remix vite:build`

Previously, Remix generated various options for building with esbuild and built it, but I will write about how it is done in the Vite version. This can also be seen by reading from vite.config.ts in the same way as remix vite:dev.

import {
  vitePlugin as remix,
  cloudflareDevProxyVitePlugin as remixCloudflareDevProxy,
} from "@remix-run/dev";
import { defineConfig } from "vite";
import tsconfigPaths from "vite-tsconfig-paths";

export default defineConfig({
  plugins: [
    remixCloudflareDevProxy(),
    remix(),
    tsconfigPaths()
  ],
});

Here we set something called vitePlugin as a Vite plugin. This is the main body that contains the settings for building.

https://github.com/remix-run/remix/blob/df0a668d416014f19313419dc7701ddcbe4ee312/packages/remix-dev/vite/plugin.ts#L530

This will return multiple Vite plugin configurations for building Remixes. Among them, the following four are the ones you should read if you want to know about build-related processes.

name: "remix": which returns the Vite build config settings
name: "remix-virtual-modules": Regenerates server-side code to build only Remix server-side code.
name: "remix-dot-server": Validates Remix's .server file from being mixed into client code.
name: "remix-dot-client": Validates Remix's .client file in server code.

So, if you want to customize the options for building with Vite, you can also change the return value of name: "remix" that is returned at the beginning of the array.
If you have been confused about not being able to change the build settings of esbuild until now, please try changing to the Vite version, as you can change the build as you like.

This will build it with Vite and generate client code and server code in build/client/ and build/server/.

By the way, in order to build for the server, I am doing something quite clumsy to build the files in app/routes as one file.

https://github.com/remix-run/remix/blob/df0a668d416014f19313419dc7701ddcbe4ee312/packages/remix-dev/vite/plugin.ts#L704-L750

As you can see, the build method has changed in the Vite version, not only for Cloudflare.

Build for Cloudflare Pages

Just having the results built with Vite will not work with Cloudflare Pages. If you want to perform server-side processing, you will need a function called Cloudflare Pages Functions that links Cloudflare Pages and Cloudflare Workers.

In the previous version, functions/[[path]].js, which is Cloudflare Pages Functions, was generated by building with Remix. The Vite version hasn't gotten this far yet.

However, the Vite version provides a file called functions/[[path]].ts from the beginning.

import { createPagesFunctionHandler } from "@remix-run/cloudflare-pages";

// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore - the server build file is generated by `remix vite:build`
// eslint-disable-next-line import/no-unresolved
import * as build from "../build/server";

export const onRequest = createPagesFunctionHandler({ build });

In the previous version, it was used as a place to place the JavaScript result of the build, but in the Vite version, TypeScript has been provided from the beginning, and build/server/index.js is loaded in it. wrangler (pages) recognizes a directory called functions/ as Cloudflare Pages Functions, but it supports not only JavaScript but also TypeScript.

https://developers.cloudflare.com/pages/functions/typescript/

So, by doing this, we changed the method to generate Cloudflare Pages Functions that include the built JavaScript of Remix's server-side processing when running wrangler pages dev and wrangler pages deploy.

Note that after all, you need to remix vite:build before start(wranlger pages dev) will work.

As you can see, Cloudflare Pages compatible with the Vite version of Remix was made possible not only by Remix, but also by Wrangler and Vite.
Although there are some issues, the development experience with Remix has been further improved by moving to Vite, so please give it a try.

Allow Cloudflare D1 to be used like an O/R mapper with type-safe

chimame — Sun, 15 Jan 2023 17:27:39 +0000

Cloudflare D1 (hereafter: D1) was released as an alpha version last year, but it is only an alpha version, so I feel that it does not scratch the itch. It is not only the functionality as a DB, such as transactions, but also the writing style as a program is a bit more complicated. (The main reason for this is that I have actually written SQL.)

The main topic of this article is to try to deal with the writing style of D1, especially SQL and types.

Prerequisites
https://github.com/chimame/workers-remix-d1

The complete program used in this article can be found at here.
In addition to D1, Cloudflare Pages is also used.
The program to deploy to Cloudflare is Remix.
I will describe D1 and the programs around it, but I will not describe everything in detail.

Summary.

kysely kysely-codegen kysely-d1 allows access to D1 while using SQL Builder.
It is even easier to use D1 than the standard DB.prepare(<SQL>).all() because the type is automatically populated.

What is Cloudflare D1?

https://www.cloudflare.com/press-releases/2022/cloudflare-announces-d1-first-integrated-database/

Unlike other databases on the market, Cloudflare D1 will use Cloudflare's global network to optimize a businesses' database by locating it as close as possible to their customers, providing the fastest possible experience to users.

In layman's terms, you can build your DB on the edge (CDN, etc.) and have it quickly accessible from applications running on the edge. Edge computing has the advantage of being close to the client (access source), but when accessing the DB, the program running on the edge will eventually access the DB behind it. In this case, the need for DB access diminishes the advantage of being close to the client. Therefore, it is inevitable that if a program runs on the edge of edge computing, the DB should also be on the edge. D1 is the solution to this problem.

Access program to D1

standard version

The following is a program to access D1.

const data = await env.DB.prepare('SELECT * FROM User').all()
console.log('data.results', data.results)
// #=> [
// { id: <number>, email: <string>, name: <string | null> }
// { id: <number>, email: <string>, name: <string | null> }
// ] type will be unkown[] | unkown

What do you think? I guess what I mean is like this

You need to write SQL.
You need to specify the type in generics

The "need to write SQL" is a bit troublesome when using O/R mappers such as Prisma or TypeORM, for example. But even if you allow it, the trouble with writing code in TypeScript is that the return value (data.results) does not have a type. In other words, the type cannot be assigned without interpreting the SQL to be executed by prepare. So, in the above code, you need to define the type of the return value yourself with generics in the all function.

const data = await env.DB.prepare('SELECT * FROM User').all<{
  id: number,
  email: string,
  name: string | null
}>()

Currently it's Alpha version, so it could be improved, but I still don't want to write double SQL and type.

Kysely version

There is a TypeScript SQL builder called Kysely.

https://koskimas.github.io/kysely/

Kysely supports PostgreSQL, MySQL, and SQLite as standard. However, Kysely also requires generic type specification for stand-alone use.

As you can see in this example, you define the database table list to the columns by yourself. I would also like to somehow omit this generics specification.

Automatic generation of table and column types

So use Kysely-codegen.

https://github.com/RobinBlomberg/kysely-codegen

As the name suggests, Kysely-codegen generates types for Kysely from the database. For example, for local development using D1, the SQLite database file is located at

<project root>/.wrangler/state/d1/<database Binding name>.sqlite3

So if you use kysely-codegen to do it like this

$ DATABASE_URL=. /.wrangler/state/d1/DB.sqlite3 npx kysely-codegen

type, which makes it very easy to use Kysely with that type.

import type { DB } from 'kysely-codegen'
import { Kysely, SqliteDialect } from 'kysely'
import Database from 'better-sqlite3'

const db = new Kysely<DB>({
  dialect: new SqliteDialect({
    database: new Database('. /.wrangler/state/d1/DB.sqlite3')
  })
})

Using Kysely with D1

Now that the table and column types have been automatically generated, we need a way to use Kysely in D1. So now we will use Kysely-D1.

https://github.com/aidenwallis/kysely-d1

This is an adapter that allows Kysely to handle D1. By the way, there are several adapters for Kysely.

https://github.com/serverless-stack/kysely-data-api

https://github.com/depot/kysely-planetscale

Kysely-D1 allows Kysely to handle D1, and the code looks like this

//const data = await env.DB.prepare('SELECT * FROM User').all<{
// id: number,
// email: string,
// name: string | null
//}>()

import type { DB } from 'kysely-codegen'
import { Kysely } from 'kysely'
import { D1Dialect } from 'kysely-d1'

const client = new Kysely<DB>({ dialect: new D1Dialect({ database: env.DB }) })
const users = await client.selectFrom('User').selectAll().execute()

console.log('users', users)
// #=> [
// { id: <number>, email: <string>, name: <string | null> }
// { id: <number>, email: <string>, name: <string | null> }
// ]

What do you think? Doesn't this make it look a lot more like an O/R mapper? The type completion is also auto-completed, so I think it's easier to use now.

Conclusion

How was it? Whether you use Kysely or Prisma, my feeling is that you are writing SQL, so it is easier to automatically define SQL results as a type.

Of course, D1 is still in its alpha version, so there is a possibility that they will fix these points. If you want to play with D1, please refer to this page.

By the way, I have also made a repository that uses sqldef as a migration tool to make migration as easy as possible, so please take a look at it.