DEV Community: Ted Klingenberg

The New Transparent RPC for JavaScript

Ted Klingenberg — Mon, 23 Oct 2023 12:30:00 +0000

Getting data from a JavaScript server is too complicated. I'm not saying that it's difficult but complicated. We have many options to do so but they all require us to rethink how we acquire that data. Let's say that I have some function defined on the server:

function createProfile({ name, email, password, photo }) {
    // ... (implementation irrelevant)
    return newProfile.id
}

This function does exactly what I want. I haven't yet thought about the server, I just know that I want to create a profile and get the ID back. Let's consider how this function gets to the client.

We could set up an HTTP server but we'll need to wrap our function in a route, mapping our function to HTTP-specific verbs and errors. We could try GraphQL but we'll need to define a schema for our function in its IDL. We could try gRPC but we'll need to create protocol buffers, implement them, and generate a client. We could try tRPC but we'll need to wrap our function in its procedures and router. But even then, most of these options require special processing for files or additional setup for a type-safe client or a lot more work to support real-time events from the server.

What's the Alternative?

None of these options are necessarily difficult and there are additional frameworks to make them easier but they all add a lot of wrappers to the function that I want to call when I just want to call the function itself. And this is just one function, what if I have hundreds?

If I have a server written in JavaScript and the client is JavaScript, shouldn’t I be able to call the function, like it’s JavaScript?

createProfile({
    name: "Ted Test",
    email: "ted@example.com",
    password: "redacted :)",
    photo: new File([...], "headshot.jpg")
})

That's what we're going to do today. We'll use an open-source library called Prim+RPC: a thin layer between the server and client of your choice that translates function calls and results so that you can simply call your server functions on the client as intended.

💡 This article is a much shorter adaptation of the setup guide on the Prim+RPC website. If you'd like a more in-depth look at this project, check out the full guide.

Star on GitHub

Let's Get Started!

I've set up a starter project for us, so we don't have to start from scratch. It is a monorepo composed of two modules: a Node server where our function will be defined and a client that will call the function, both serialized using Prim+RPC's plugins for the Fetch API.

You can download the project with the command below or download it directly:

npx giget@latest "gh:doseofted/prim-rpc-starter#follow-along"

It's also available and easy to get started on StackBlitz:

stackblitz.com

You can start the project by running npm install to install dependencies and then npm run dev to start both the server and client parts of the project. The client will be available at http://localhost:3000.

We'll be greeted with a message "Not implemented". Let's fix that!

Server Setup

Let's set up the server first. First things first: let's define a function. We'll work with something simple to get started. Add this function to server/module.ts:

export function sayHello(x = "Backend", y = "Frontend") {
    return `${x}, meet ${y}.`
}
sayHello.rpc = true

This is just a regular JavaScript function. The only thing that we added is a property .rpc that signals to Prim+RPC that we want to expose this function as RPC. Since Functions in JavaScript are also Objects, this is valid syntax (we could also omit this property and add our function to an allow-list).

Moving on to the server framework itself, replace the contents of server/index.ts with the following. I'll explain what this does:

import { createPrimServer } from "@doseofted/prim-rpc"
import { primFetch } from "@doseofted/prim-rpc-plugins/server-fetch"
import { createServer } from "node:http"
import { createServerAdapter } from "@whatwg-node/server"
import * as module from "./module"

const prim = createPrimServer({ module })
function postprocess(res: Response) {
    res.headers.set("access-control-allow-origin", "http://localhost:3000")
    res.headers.set("access-control-allow-headers", "content-type")
}
const fetch = primFetch({ prim, postprocess })

const fetchAdapter = createServerAdapter(fetch)
createServer(fetchAdapter).listen(3001)
console.log("Prim+RPC is available at http://localhost:3001/prim")

export type Module = typeof module

First, we pass the module containing our function to createPrimServer(): a framework-agnostic utility to receive RPC and respond with RPC results. We then pass the Prim+RPC server to a method handler, primFetch(), which wraps our RPC into Fetch API Request and Response objects. We also add two necessary CORS headers on every Request so that our client in the browser is allowed to request resources from the server.

💡 If you are running this example from Stackblitz, you may need to change the access-control-allow-origin header value to https://localhost:3000 if you experience any issues running the project (using secure HTTP).

The returned fetch utility is a function that takes a Request and returns a Response, as expected by server frameworks like Deno and Bun. Since Node doesn't yet support this natively, we use an adapter, createServerAdapter(), for Node's createServer utility. Now we can expose our function over HTTP, using port 3000. As a last step, we export the type of our module which will later become a type import on the client (exposing types only, not the actual code).

We did it. We're ready to move on to the client. As a sanity check, we can test this out now by sending a simple request. With the server running, try out this command :

curl "http://localhost:3001/prim/sayHello?0=Backend&1=Terminal"

Our Prim+RPC server is now available at http://localhost:3000/prim (you can also change the path prefix if you'd like).

Client Setup

The client is very easy to set up. Replace the contents of client/prim.ts with the following. I'll explain what this does:

import { createPrimClient } from "@doseofted/prim-rpc"
import { createMethodPlugin } from "@doseofted/prim-rpc-plugins/browser-fetch"
import type { Module } from "../server"

export const client = createPrimClient<Module>({
    endpoint: "http://localhost:3001/prim",
    methodPlugin: createMethodPlugin(),
})
export default client

We have set up the Prim+RPC client, a framework-agnostic tool for sending RPC and receiving RPC results. We pass a type parameter to our client with the type of our module: the Prim+RPC client will apply RPC-specific transformations to our types. Note that we don't pass the module code itself, only the TypeScript types. We then pass the server address to the client as well as a method plugin that will turn our function calls into Requests for the Fetch API. We export this client to be used throughout our app.

That's all there is to it. We can now use this client to call our function. Replace the contents of client/index.ts with the following:

import { client } from "./prim"

const greeting = await client.sayHello()
console.log(greeting) // "Frontend, meet Backend."

const app = document.getElementById("app")
if (app) app.innerText = greeting

We didn't need to generate code for a client or wrap our function call in some request specific to the client, we just called the function and received a result.

Since our server and client are in two separate environments and it may take a (very short) amount of time to resolve, our result is wrapped in a Promise so as not to block execution on the main thread (which means we await the result). But the Prim+RPC client automatically transforms our Module types for us, so we know.

That Was Easy

Now, we can add more functions to the server and we don't need any more setup to call those functions on the client. It's just ready to be called.

We could stop here. But we can do much more with Prim+RPC: we can support file uploads and downloads, additional types, add validation, and even pass server context to our functions when needed.

Read on to learn how we can use these features.

⭐ Prim+RPC is an open-source project, currently in prerelease. If you find the library useful, consider giving it a star on GitHub and sharing it with others.

Star on GitHub

Adding Validation

When calling a function over the Internet, we don't know what could be passed to the server. Prim+RPC doesn't validate arguments by default but we can add this easily. Let's validate using Zod as an example. First, we'll install it on the server:

cd server
npm add zod

Let's modify our server/module.ts so we validate and overwrite the given arguments:

import { z } from "zod"

export function sayHello(x = "Backend", y = "Frontend") {
    ;[x, y] = z.tuple([z.string(), z.string()]).parse([x, y])
    return `${x}, meet ${y}.`
}
sayHello.rpc = true

And now we have validation. If you'd like to ensure functions are never defined without validation, check out Prim+RPC's security guide for more details.

File Support

Prim+RPC can support files as arguments and return them to the caller. With the Fetch plugin, we don't even have to do any extra work. It works out of the box! Let's demonstrate this with an example that converts a Markdown file to HTML, using micromark. First, we'll install it on the server:

cd server
npm add micromark

And replace server/module.ts with the following:

import { micromark } from "micromark"
import { File } from "node:buffer"

export async function markdownToHtml(markdownFile: File | string) {
    const markdown = typeof markdownFile === "string" ? markdownFile : await markdownFile.text()
    const html = micromark(markdown)
    return new File([html], "snippet.html", { type: "text/html" })
}
markdownToHtml.rpc = true

On the client, we just call it. Update client/index.ts:

import { client } from "./prim"

const markdown = "[**Backend**, meet **Frontend**.](https://prim.doseofted.me/)"
const htmlFile = await client.markdownToHtml(markdown)
console.log(htmlFile.name, htmlFile instanceof File)

const app = document.getElementById("app")
if (app) app.innerHTML = await htmlFile.text()

We could also access returned files from a URL, for easy file downloads. Try the following command or visit the given URL in your web browser:

curl "http://localhost:3001/prim/markdownToHtml?0=_Hello%20there_"

Extended Types

Prim+RPC, by default, can support arguments and return values serializable by JSON (with files processed separately). But what if we want more? Libraries like superjson can add support for additional types like Map, Set, Date, BigInt, and more. And Prim+RPC supports custom serialization like superjson! Let's install it in our project:

cd server
npm add superjson
cd ../client
npm add superjson

Now we can change the .jsonHandler option in server/index.ts. Replace the contents of this file with the following:

import { createPrimServer } from "@doseofted/prim-rpc"
import { primFetch } from "@doseofted/prim-rpc-plugins/server-fetch"
import * as module from "./module"
import { createServer } from "node:http"
import { createServerAdapter } from "@whatwg-node/server"
import jsonHandler from "superjson"

const prim = createPrimServer({ module, jsonHandler })
function postprocess(res: Response) {
    res.headers.set("access-control-allow-origin", "http://localhost:3000")
    res.headers.set("access-control-allow-headers", "content-type")
}
const fetch = primFetch({ prim, postprocess })

const fetchAdapter = createServerAdapter(fetch)
createServer(fetchAdapter).listen(3001)
console.log("Prim+RPC is available at http://localhost:3001/prim")

export type Module = typeof module

Serialization and deserialization go hand-in-hand so let's add this to the client too. Replace the contents of client/prim.ts with the following:

import { createPrimClient } from "@doseofted/prim-rpc"
import { createMethodPlugin } from "@doseofted/prim-rpc-plugins/browser-fetch"
import type { Module } from "../server"
import jsonHandler from "superjson"

export const client = createPrimClient<Module>({
    endpoint: "http://localhost:3001/prim",
    methodPlugin: createMethodPlugin(),
    jsonHandler,
})
export default client

Now can create functions that use these new supported types. Let's try it out. Replace server/module.ts with:

export function whatIsDayAfter(day: Date) {
    return new Date(day.valueOf() + 1000 * 60 * 60 * 24)
}
whatIsDayAfter.rpc = true

And call it in client/index.ts:

import { client } from "./prim"

const tomorrow = await client.whatIsDayAfter(new Date())
console.log(tomorrow, tomorrow instanceof Date)

const app = document.getElementById("app")
if (app) app.innerText = tomorrow.toDateString()

See the JSON Handler documentation to learn how to set up other popular JSON alternatives.

Support Callbacks

Prim+RPC can support callbacks on your function as well. We just need to set up a callback handler. While functions return once, callbacks on a function can fire multiple times so we need to maintain a connection to the server. For this, we'll use a callback handler backed by a WebSocket connection.

First, let's install ws which will act as our WebSocket server:

cd server
npm add ws
npm add -D @types/ws

Replace server/index.ts with the following. And stick with me: this is one-time setup and we don't have to touch WebSockets again after this:

import { createPrimServer } from "@doseofted/prim-rpc"
import { primFetch } from "@doseofted/prim-rpc-plugins/server-fetch"
import { createCallbackHandler } from "@doseofted/prim-rpc-plugins/ws"
import * as module from "./module"
import { createServer } from "node:http"
import { createServerAdapter } from "@whatwg-node/server"
import { WebSocketServer } from "ws"
import jsonHandler from "superjson"

const wss = new WebSocketServer({ noServer: true })
const callbackHandler = createCallbackHandler({ wss })
const prim = createPrimServer({ module, jsonHandler, callbackHandler })
function postprocess(res: Response) {
    res.headers.set("access-control-allow-origin", "http://localhost:3000")
    res.headers.set("access-control-allow-headers", "content-type")
}
const fetch = primFetch({ prim, postprocess })

const fetchAdapter = createServerAdapter(fetch)
const server = createServer(fetchAdapter).listen(3001)
console.log("Prim+RPC is available at http://localhost:3001/prim")

server.on("upgrade", (request, socket, head) => {
    wss.handleUpgrade(request, socket, head, ws => {
        wss.emit("connection", ws, request)
    })
})

export type Module = typeof module

Are you still with me? We have just set up a WebSocket connection and passed that WebSocket server to Prim+RPC. Now the server can handle callbacks on your functions!

However, the client doesn't yet know how to tell the server that it has callbacks. Let's do that now using the WebSocket callback plugin:

import { createPrimClient } from "@doseofted/prim-rpc"
import { createMethodPlugin } from "@doseofted/prim-rpc-plugins/browser-fetch"
import { createCallbackPlugin } from "@doseofted/prim-rpc-plugins/browser-websocket"
import type { Module } from "../server"
import jsonHandler from "superjson"

export const client = createPrimClient<Module>({
    endpoint: "http://localhost:3001/prim",
    methodPlugin: createMethodPlugin(),
    callbackPlugin: createCallbackPlugin(),
    jsonHandler,
})
export default client

And now we can use callbacks on our functions. Let's create a function that uses them in server/module.ts:

export function typeMessage(message: string, typed: (letter: string) => void) {
    let timeout = 0
    const letters = message.split("")
    for (const letter of letters) {
        setTimeout(() => typed(letter), ++timeout * 300)
    }
}
typeMessage.rpc = true

This will type the message that we give it, one-by-one. Not a great use of server resources but a good demo. Let's try this on the client (client/index.ts):

import { client } from "./prim"

const app = document.getElementById("app")
if (app) {
    client.typeMessage("Hello!", letter => {
        app.innerText += letter
    })
}

And now we see the message typed out on the page!

Pass Server Context

Up to this point, we have not really touched the server inside of our functions. Instead, we've passed everything that our function needs as arguments. However, we can't always just ignore the server: it contains information that we may need that doesn't have direct comparisons to calling a function in JavaScript. But we can still access the server context from our functions. And that's what we're going to do next.

In this example, we’ll set a secret cookie from the client that is required to access our secret function. Without the cookie: no function access. However, our function won’t have to touch the cookie at all.

First, let's install a tiny helper package:

cd server
npm add cookie
npm add -D @types/cookie

And now we can use this package on our server. We'll define a contextTransform option that will be given the Request object (from the Fetch API) for every function call that we make to the server. The returned value from this function will become the context (this value) of our function.

Replace server/index.ts with the following:

💡 If you are running this example from Stackblitz, you will need to change the access-control-allow-origin header value to https://localhost:3000 for this example to work in the browser (using secure HTTP).

import { createPrimServer } from "@doseofted/prim-rpc"
import { primFetch } from "@doseofted/prim-rpc-plugins/server-fetch"
import { createCallbackHandler } from "@doseofted/prim-rpc-plugins/ws"
import * as module from "./module"
import { createServer } from "node:http"
import { createServerAdapter } from "@whatwg-node/server"
import { WebSocketServer } from "ws"
import jsonHandler from "superjson"
import { parse, serialize } from "cookie"

const cookieOpts = { httpOnly: true, sameSite: "none", secure: true } as const
function contextTransform(req: Request, res?: { headers: Headers }) {
    const secret = "can't-touch-this"
    return {
        setSecret(given: string) {
            res?.headers.set("set-cookie", serialize("secret", given, cookieOpts))
        },
        get allowed() {
            return secret === parse(req.headers.get("cookie") ?? "").secret
        },
    }
}
export type ServerContext = ReturnType<typeof contextTransform>

const wss = new WebSocketServer({ noServer: true })
const callbackHandler = createCallbackHandler({ wss })
const prim = createPrimServer({ module, jsonHandler, callbackHandler })
function postprocess(res: Response) {
    res.headers.set("access-control-allow-origin", "http://localhost:3000")
    res.headers.set("access-control-allow-headers", "content-type")
    res.headers.set("access-control-allow-credentials", "true")
}
const fetch = primFetch({ prim, postprocess, contextTransform })

const fetchAdapter = createServerAdapter(fetch)
const server = createServer(fetchAdapter).listen(3001)
console.log("Prim+RPC is available at http://localhost:3001/prim")

server.on("upgrade", (request, socket, head) => {
    wss.handleUpgrade(request, socket, head, ws => {
        wss.emit("connection", ws, request)
    })
})

export type Module = typeof module

Inside of the contextTransform function, we return setSecret() and allowed which will be used from our function. We have also updated the CORS headers to allow credentials so that the browser can utilize the cookies we set. Using TypeScript, we get that return type and assign it a type ServerContext. We'll pass this as a type hint to our function.

❗ This is of course a demo. In a real application, you will want to use some form of cryptography.

Replace server/module.ts with the following:

import type { ServerContext } from "./index"

export function secretMessage(this: ServerContext, secret?: string) {
    if (secret) {
        this.setSecret(secret)
        return ""
    }
    if (!this.allowed) throw new Error("No secret, no entry.")
    return "Access granted! The answer is 42."
}
secretMessage.rpc = true

Notice that we only accessed the functions that we defined in the server's context so we didn't even have to touch the cookie at all. The this parameter is only a type hint and is not an argument that we have to pass: Prim+RPC will handle that for us.

Now we can make one small change to the client. Since we're passing cookies down to the client, we must set the credentials option of the fetch function to “include” so that cookies can be set properly.

Let's do that real quick in client/prim.ts:

import { createPrimClient } from "@doseofted/prim-rpc"
import { createMethodPlugin } from "@doseofted/prim-rpc-plugins/browser-fetch"
import { createCallbackPlugin } from "@doseofted/prim-rpc-plugins/browser-websocket"
import type { Module } from "../server"
import jsonHandler from "superjson"

export const client = createPrimClient<Module>({
    endpoint: "http://localhost:3001/prim",
    methodPlugin: createMethodPlugin({ credentials: "include" }),
    callbackPlugin: createCallbackPlugin(),
    jsonHandler,
})
export default client

And now we can call our function in client/index.ts:

import { client } from "./prim"

await client.secretMessage("can't-touch-this")

const app = document.getElementById("app")
if (app) app.innerText = await client.secretMessage()

Since we have set the secret in a cookie, we don't technically need to set the cookie anymore. If we just comment out that line:

import { client } from "./prim"

// await client.secretMessage("can't-touch-this")

const app = document.getElementById("app")
if (app) app.innerText = await client.secretMessage()

You'll find that we can still see the secret because we had already set the secret before and no longer need to pass any arguments to our function. It's handled in the server context!

This can be a powerful tool for setting up authentication, adding redirects, or otherwise integrating with the server of your choice.

Summary

All of the frameworks that I mentioned in the beginning have their place. But when I'm getting started on a new project, how messages get passed from server to client is far from the first thing on my mind. It's important but it shouldn't dictate the logic happening on my server or restrict what I want to do.

Prim+RPC provides a powerful library for calling functions remotely and remains invisible once it is set up. I can define a function on the server of my choice and call that function on the client, all using JavaScript. Yet requests are simple enough to make outside of JavaScript as well.

⭐ Thank you for following along with me. If you like this method of retrieving data from the server, consider giving Prim+RPC a star on GitHub.

Star on GitHub

Try adding your own functions to this project, maybe build out a simple API. If you get stuck, there are several examples on the Prim+RPC website as well as this post.

Easy RPC with Nuxt: Making a Todo App

Ted Klingenberg — Tue, 27 Jun 2023 13:27:30 +0000

We're going to make a simple to-do app with Nuxt. Except, instead of manually defining server routes and fetching them on the client, we're going to use Prim+RPC to define functions on Nuxt's server and simply call those functions directly from the client (without actually importing them).

Finished Tutorial Code : doseofted/todo-nuxt-prim-rpc-demo (Github)
Bare Bones Example : doseofted/prim-rpc-examples (Github)

The Problem

There are many times when I need to call a function on the server from the client, without the ceremony of setting up an HTTP server. When I'm just playing around with an idea things like serialization, type generation, file upload support, HTTP methods and status codes just aren't yet something that I'm thinking about. Luckily Nuxt provides utilities for making HTTP calls easier. Using Nuxt's server API routes and the useFetch composable I can alleviate some of my HTTP hassles. The syntax for defining routes is clean and easy to use. When combined with the useFetch composable, my routes are partially typed and I can see the return type from the client. Here's an example:

// Server: @/server/api/hello.ts
export default defineEventHandler(event => {
  const { name = "world" } = getQuery(event)
  return { hello: String(name) }
})

// Client: somwhere inside of @/pages/index.vue
const { data: hi } = useFetch("/hello", { query: { name: "world" } })

There are a few caveats though:

Each HTTP route in Nitro requires its own file. The exception is using a "catch-all" route but then I'll need to check every URL.
TypeScript support for routes only works as long as I'm using Nuxt's server. Queries are not yet typed at the time of writing.
I'm still writing routes specific to an HTTP server when all I really want to do is call a function defined on the server from the client.

There is now an easy way to do this not only in Nuxt but in a way that works with whatever your favorite framework happens to be. It's called Prim+RPC and it provides a method of accessing server-defined functions on the client without additional wrappers. Let's see what our last example looks like with Prim+RPC:

❗ Prim+RPC is prerelease software at the time of writing, not intended yet for production. It could be unstable and functionality might prior to full release.

// Server: @/functions/index.ts
export function hello (name = "world") {
  return { hello: String(name) }
}
hello.rpc = true

// Client: somwhere inside of @/pages/index.vue
const { data: hi } = useAsyncData(() => backend.hello("world"))

Our server call now just appears to be a regular JavaScript function. We can use the client, which I've named "backend", to call that function. The return value is whatever we returned, just wrapped in a Promise (that's why I'm also using useAsyncData). You'll also notice that the parameters are now typed on the client!

While it appears that we are calling the function, the client is intercepting the function name and arguments with a JavaScript Proxy and then composing a remote procedure call (RPC) to the server for us. The Prim+RPC server and client handle the server/client dance so that we can just call our functions.

Let's See It In Action

Let's demonstrate this with a short tutorial. I've created a basic todo app with everything almost ready to go: server logic is ready and the client app has already been created. The only thing missing: the server and client can't communicate yet.

Let's learn how to wire them up in this tutorial. You can follow along by downloading the unfinished demo like so:

 npx giget@latest "gh:doseofted/todo-nuxt-prim-rpc-demo#starter"

This command will download a copy of the demo repository on Github. If you'd like to download a copy of the finished code, you can remove #starter from the command above to download the finished demo. Open this project in your favorite code editor.

Note that this demo uses Node 18+. If you're using a lower version, you may change your Node version easily using a version manager like nvm.

First, let's set up the project by copying the example environment file. In this file, you'll find a command that can be used to generate the value of DATABASE_URL. This is the connection string used by Prisma to communicate with our SQLite file.

cp .env.example .env
echo "DATABASE_URL=\"file:$(pwd)/data/dev.db?connection_limit=1\"" >> .env

Now we can start the project in three commands. We'll run them in sequence: 1. install dependencies 2. prepare the SQLite database and 3. start the project.

npm install && npm run migrate:dev && npm run dev -- --open

A browser window will automatically open. You'll find that we don't have any tasks just yet. This is because the app cannot yet communicate with the server.

All of the functions that should be called on the server are located in the module found at @/functions/index.ts (@ representing the root of the project). This is primarily a series of functions to create todo items. In Nuxt alone, we would expose functions through a series of Nitro server routes. In our example, we'll set up Prim+RPC using a single API route.

Create a file at the path: @/server/api/[...].ts (this is a "catch-all" route)

import { createPrimServer } from "@doseofted/prim-rpc";
import { defineH3PrimHandler } from "@doseofted/prim-rpc-plugins/h3";

const prim = createPrimServer({
  module: import("../../functions"),
  prefix: "/api/prim",
});

export default defineH3PrimHandler({ prim });

In this file we've created a Prim+RPC "server." This is a framework-agnostic utility that transforms an RPC (typically a JSON object) into a function call and the return value into an RPC response. I put "server" in quotes because it doesn't serve anything alone. We need to define the channel over which RPC travels. defineH3PrimHandler() creates an unjs/h3 route that is compatible with Nuxt.

By default, functions aren't allowed to be called unless marked as allowed. We can do so by adding a .rpc property to each function.

Open @/functions/index.ts and add this property to the hello() function.

export * as todo from "./todo";

export function hello() {
  console.log("Hello server!");
  return "Hello client!";
}
hello.rpc = true;

export type { TodoItem } from "./todo/schema";

While we have not set up the client yet, we can already call the hello function! Try visiting this URL with Nuxt running:

curl --request GET \
  --url http://localhost:3000/api/prim/hello

You'll be greeted with a result both from your request and in the server's console. The module we gave the server is being served at http://localhost:3000/api/prim. Typically requests are made over POST using a JSON body but we don't have to concern ourselves with the details because the next step is setting up the Prim+RPC client that will automatically make this request for us.

Let's create the client now. Create a file at the path @/composables/backend.ts :

import { createPrimClient } from "@doseofted/prim-rpc";
import { createMethodPlugin } from "@doseofted/prim-rpc-plugins/browser";

export const backend = createPrimClient<typeof import("@/functions")>({
  module: process.server ? import("@/functions") : null,
  endpoint: "/api/prim",
  methodPlugin: createMethodPlugin(),
  clientBatchTime: 15,
});

export type { TodoItem } from "@/functions";

We've just created a composable named backend that we can call all throughout our Nuxt app. It provides a method to call our functions remotely on the client (and directly on the server). We can test this out by adding a function call on the index page of our app.

Open @/pages/index.vue and replace this line at the top of the file:

// replace this line
tryOnMounted(() => console.log("Not implemented"))
// with this line
tryOnMounted(() => backend.hello().then(console.log))

Now open http://localhost:3000 in your browser, right-click and inspect the page. If you open the console, you'll see our greeting in the console for the client. If you check the console where the server is running, you'll find the greeting for the server. Better yet, this function is typed with TypeScript.

That's all there is to it! Prim+RPC is now set up and we can start making our functions available to the client. Next up, we'll wire up all of our todo-related functions to the user interface.

💡 Download the fullstack framework example for a barebones example of setting up a Nuxt application with Prim+RPC. You can find a simpler example running in your browser on Stackblitz.

Let's Wire It Up!

Let's get this todo app working now. In the file @/functions/index.ts you may remember seeing this line:

export * as todo from "./todo";

If we open that file (@/functions/todo/index.ts) we'll find several functions to interact with our todo items. Just like the hello() function we looked at earlier, we'll need to export the functions that we need and add a .rpc property to each function. Let's do this now to the following functions:

export async function count() { /* ... */ }
count.rpc = true

export async function find(todoId: TodoItemId) { /* ... */ }
find.rpc = true

export async function check(todoId: TodoItemId) { /* ... */ }
check.rpc = true

export async function list(page?: z.infer<typeof list.params>["0"], pageSize?: z.infer<typeof list.params>["1"]) { /* ... */ }
list.rpc = true

export async function create(todo: HTMLFormElement | z.infer<typeof create.todo>) { /* ... */ }
create.rpc = true

export async function update(todo: HTMLFormElement | z.infer<typeof update.todo>) { /* ... */ }
update.rpc = true

Great! Now these functions are ready to be used by the client. Open up @/pages/index.vue. The first thing that we want to do is get the initial page data, given in our useAsyncData function. Let's replace the sample data with the actual data:

Find and replace the line below in @/pages/index.vue

// replace this line
const { data, refresh: refreshTodos } = useAsyncData(() => Promise.all([
  [{ name: "N/A", desription: "N/A", photo: "", id: -1 }],
  0
]), { watch: [page] })

// with this line
const { data, refresh: refreshTodos } = useAsyncData(() => Promise.all([
  backend.todo.list(page.value, pageSize),
  backend.todo.count()
]), { watch: [page] })

Save this file and open up your browser (reload if needed).

We're getting somewhere! However, when we tap on a task, we don't see the task that we tapped. Instead, we see "N/A". Let's fix that now.

🤔 If you're wondering where these tasks came from, you can check @/functions/prisma.ts where we run a function on startup to populate an empty database with fake data in development.

Open up @/pages/index/[id].vue and replace the following line:

// replace this line
const { data: page, refresh } = useAsyncData(async () => ({
  name: "N/A", description: "N/A", photo: ""
}), { watch: [id] })

// with this line
const { data: page, refresh } = useAsyncData(() =>
  backend.todo.find(id.value), { watch: [id] })

Now let's go back to our browser and open up a task.

Well, there it is! We can now browse all of our tasks. Now the last step is to make it a little more interactive. Let's learn how to create and update a task.

I Have Something Todo

You may remember that our @/functions/todo/index.ts file included two methods for creating tasks: todo.create() and todo.update(). We could create a task by calling this function using our backend client with the needed arguments. Here's an example:

const myTodoItem = await backend.todo.create({
  name: "My Todo",
  description: "This needs to be done.",
  file: new File([/* ... */], "some-attachment.jpeg")
})

This works just fine and can be called from the client. We can pass all of our todo options directly to our function, including a file. The arguments we give the function from the client will be the same on the server.

But I think that we can make this even easier. If we tap the New Task button in the app, we're presented with an HTML form. What if, instead of tracking all of the variables, we just give our function the form and let it grab the data that it needs?

Let's do that. You'll notice that inside the type definitions for our create() function is an HTMLFormElement. We can listen for the form's "submit" event (when we tap the Create button) and just pass the target directly to our function.

Look for the createNewTask() function inside of @/pages/index.vue.

async function createNewTask(event: SubmitEvent & { target: HTMLFormElement }) {
  try {
    console.warn("Not implemented")
    // ...
  } catch (error) {
    console.warn(error)
  }
}

Replace console.warn("Not implemented") above with the following:

await backend.todo.create(event.target)

And now we can create new tasks in the app! The backend client has read the form, grabbed the names and values of each form field and passed it to the server as a regular JavaScript object (no more manual parsing of FormData).

Of course, a todo app isn't much without being able to check items off of the list.

Look for the following function in the same file and make the replacement given:

async function checkOffItem(id: number) {
  // replace this line
  console.warn("Not implemented")
  // with this line
  await backend.todo.check(id)
  setTimeout(refreshTodos, 150)
}

The next natural step is to check off some items on the list. You'll notice that we can now check off items individually. There's also a button at the bottom of the page named Clear that doesn't do anything just yet. Let's wire this button up too (this one is unique).

Still in the same file, replace the function given below with the following:

// replace this
async function clearAll() {
  await Promise.all(todoList.value.map(todo => console.warn("Not implemented")))
  setTimeout(refreshTodos, 150)
}

// with this
async function clearAll() {
  await Promise.all(todoList.value.map(todo => backend.todo.check(todo.id)))
  setTimeout(refreshTodos, 150)
}

Unlike our previous examples, we're not just calling the backend function once. We're calling function 10 times (10 todo items per page), all at once! Now you would think that this would send multiple HTTP requests off to the server. However, we set an option on the client earlier called clientBatchTime which specified how long to wait before sending requests. It's currently set to 15ms. Instead of sending 10 separate HTTP requests, only a single call was made with all of our function calls included. You can verify this in the Network tab of your browser's developer tools.

Check and Check!

We've now set up most of the major actions inside of the todo app and we're almost finished but not quite yet. There are several more places where you'll find a message console.warn("Not implemented"). As an exercise, see if you can call the correct corresponding backend function that should be used in place of these placeholders (but know that there's a cheat sheet available if you scroll down a bit). The type definitions will guide you so that you shouldn't even need to reference the @/functions folder at all.

You may also take a peek at the completed example to find what function calls need to be made if you get stuck. Once these remaining function calls are implemented, we will have a working todo app!

If you're not quite up to the exercise, I get that, you can find the remaining replacements below! This will get the task detail page (@/pages/index/[id].vue) all wired up and ready to go:

// @/pages/index/[id].vue

async function checkOff(event: Event & { target: HTMLInputElement }) {
  // ...
  console.warn("Not implemented")
  // replace with 
  await backend.todo.check(id.value)
  // ...
}

async function updateTask(event: SubmitEvent & { target: HTMLFormElement }) {
  console.warn("Not implemented")
  // replace with 
  await backend.todo.update(event.target)
  // ...
}

async function removeAttachment() {
  console.warn("Not implemented")
  // replace with 
  await backend.todo.update({ id: id.value, file: "delete" })
  // ...
}

Our todo app is fully functional! And we didn't really have to think about client-to-server communication. We had a JavaScript function on the server and we called it from the client. Easy.

Next Steps

Try adding some new features to the app. All of the functions are located inside of @/functions and we can access those functions on the backend client anywhere in our app. Just remember to export the functions and set the .rpc property.

If you like this method of communicating between client and server consider learning more about Prim+RPC and giving it a star on GitHub! You can find the finished code for this project on GitHub as well!

Proper RPC with Prim+RPC

Ted Klingenberg — Wed, 01 Mar 2023 14:03:00 +0000

I did not think that RPC would become a popular topic in 2023 but I'm happy to find that I was very wrong. It seems that RPC has become the topic to tackle with the introduction of many new frameworks that blur the line between the client and server. It's overwhelming even, especially for me, as someone who has been working on my own RPC framework that offers another take on how RPC should be handled in JavaScript and TypeScript.

My framework however does things a little differently: it aims to use the JavaScript language itself. That is to say, this framework allows you to write regular JavaScript functions on the server and call those functions on the client without additional wrappers, without a compiler step, with full type definitions, support for handling files, callbacks, errors, and advanced types, and the ability to support your favorite existing server and client, over the transport of your choice.

I'd like to introduce Prim+RPC: a JavaScript library that bridges otherwise incompatible JavaScript environments. The goal of this RPC library is to let developers focus less on message transport and more on the message being sent, using the frameworks and tools you already love using.

I've worked across many frameworks over the past few years on many projects and those tools have not stood still. In other words, the JavaScript ecosystem is changing every day. New tools are coming out consistently and it's incredibly exciting but staying up to date and using the latest tools can lead to framework fatigue. More specifically, changes to foundational frameworks like those responsible for server and client communication have been a pain point for me, for some time. They can sometimes be difficult to learn and set up, or they require a separate schema to be defined, or come with their own validation logic that needs to be followed, or require client regeneration on API changes, or they enforce a paradigm shift in how we think about defining APIs.

Each new tool has its unique set of features designed to make API development easier. And it truly can be easier once that framework has been learned and is embraced by a community. It's an exciting time to be developing with JavaScript because these tools are introducing many new features that are intended to make development easier. However, these server frameworks also sometimes enforce new ideas that go beyond transport such as validations, schemas, generated clients, and wrappers around the code that I actually want to call on the server from the client.

With all of the innovations in these frameworks, one thing has stayed consistent in my search for new server/client frameworks: my need to call a function on the server and get the result of that function on the client.

So that's the library I set out to create. A framework that, once set up, allows me to write a regular JavaScript function and call that function directly or, at least, appear to the client as if it's calling the function directly. The result is fairly close to that intention. I can write plain JavaScript, using my favorite libraries, and allow Prim+RPC to send function calls and receive results.

Let's get straight to an example. Let's say that I just have a function on a server that I want to make available to the client.

// this is the function that we want to call on the server
function sayHello(x: string, y: string) {
    return `${x}, meet ${y}.`
}

// this is how we want to call the function on the client
const greeting = sayHello("Backend", "Frontend")

Prim+RPC makes this possible without introducing a lot of additional wrappers. Let's see how with a real demo. In this example, we set up the Prim+RPC server and client in a single file for testing purposes: this way we can compare the result on the client with the result of the actual function call. Our example function will just return a simple greeting but we could define whatever function we'd like here. Let's see how we can call this function in Prim+RPC:

import { createPrimClient, createPrimServer, testing } from "@doseofted/prim-rpc"

// write a regular JavaScript function
function sayHello(x: string, y: string) {
    return `${x}, meet ${y}.`
}
sayHello.rpc = true // expose function to Prim+RPC

// set up plugins for your transport of choice (this is a demo)
const plugins = testing.createPrimTestingPlugins()

// pass your functions and transport plugins to the server
const module = { sayHello }
const { methodHandler, callbackHandler } = plugins
const server = createPrimServer({ module, methodHandler, callbackHandler })

// pass your transport plugins to the client
const { methodPlugin, callbackPlugin } = plugins
const client = createPrimClient<typeof module>({ methodPlugin, callbackPlugin })

// call your function: it's automatically translated into RPC
const greeting = await client.sayHello("Backend", "Frontend")
const expected = sayHello("Backend", "Frontend")
console.log(greeting === expected) // true

In this example, a function sayHello() is given to the Prim+RPC server. The client then calls this function and receives a result. It's important to note that the client was never given this function but it can still read the result of the function through the handler plugins that we configured (method and callback handlers as they are known in Prim+RPC). The function call was automatically translated into an RPC using a JavaScript Proxy, processed by the server, and the result was parsed back into the expected format.

You can try this example yourself by running this command:

npx giget@latest gh:doseofted/prim-rpc-examples/simple-test prim-rpc-examples/simple-test

Of course, this code snippet is just a demo in a single file. We could've just called the function directly. Where Prim+RPC is useful is crossing the boundary between client and server. That's why Prim+RPC includes plugins for communication across HTTP, WebSocket, Web Workers, and can support additional transports by writing custom plugins. We could set up an HTTP server with Prim+RPC like so:

import Fastify from "fastify"
import { createPrimServer } from "@doseofted/prim-rpc"
import { createMethodHandler } from "@doseofted/prim-rpc-plugins/fastify"

// here's our regular JavaScript function again
function sayHello(x: string, y: string) {
    return `${x}, meet ${y}.`
}
sayHello.rpc = true
const module = { sayHello }

// in this example, we'll use Fastify as an HTTP server
const fastify = Fastify()
const server = createPrimServer({
    module,
    methodHandler: createMethodHandler({ fastify })
})
await fastify.listen({ port: 1234 })

// expose type definitions for use on the clinet
export type { module }

The Prim+RPC server has now been configured at http://localhost:1234/prim. Our function is now available to be called remotely. We just need to set up a client to communicate with it. Let's set up a Fetch plugin to connect the client and server.

import { createPrimClient } from "@doseofted/prim-rpc"
import { createMethodPlugin } from "@doseofted/prim-rpc-plugins/browser"
import type { module } from "./my-server"

const client = createPrimClient<typeof module>({
    endpoint: "http://localhost:1234/prim",
    methodPlugin: createMethodPlugin()
})

const greeting = await client.sayHello("Backend", "Frontend")
console.log(greeting) // "Backend, meet Frontend."

When we run this code, the client makes a request to our server at the given endpoint and receives the result from Prim+RPC without having access to the function directly. It even has full typed support with TypeScript by passing the module types (not the module itself) to the client. This is powerful because you get full code completion in your editor without generating a client or needing to define a schema specific to a framework. You can even use these types across projects and repositories by uploading these types to a registry (like NPM or Verdaccio). This could even allow you to define a public API with Prim+RPC and for users of the API to have a fully typed client without needing to download an API-specific client for JavaScript.

You can download this full example with the following command:

npx giget@latest gh:doseofted/prim-rpc-examples/client-server prim-rpc-examples/client-server

While the function we're using as an example here just prints out a greeting (sayHello()), we can do much more than that. We can pass callbacks to our function, throw errors, upload files, and more. This example only demonstrates the basics of Prim+RPC. I've written a full guide on how to get started with Prim+RPC and more examples are available on the website.

There are a lot of exciting RPC-related tools being released right now and I'm excited to add another to that list. Prim+RPC is now available as a prerelease and I aim to work toward a full release. I'm personally very excited to use this tool in my projects and I hope that you are as well. If you are excited about the project, consider giving it a star on Github and trying it out today!

I'll be writing more about Prim+RPC occasionally among other design/development topics. Follow my blog for more!