Francisco Mendes

Posted on Dec 10, 2023

Building Real-Time Next.js Apps with WebSockets and Soketi

#nextjs #react #javascript #node

Introduction

Nowadays, it is becoming increasingly common to have certain components of web applications with features that are updated in real time. And one of the things that users expect is that the integration of these functionalities is done seamlessly in a dynamic way that goes beyond the normal request-response model with data pulling strategies.

In today's article we will focus on one of my favorite trios, we will use the Next.js framework to build the web application, Drizzle to define the database schema and interact with it and perhaps the most important component of this article, Soketi.

If you have never heard of Soketi, to give you a brief overview, it is a WebSocket server that was built on top of uWebSockets.js and has great compatibility with Pusher Protocol.

The application we are going to develop today is a chat app, in which we will start by defining the database schema, we will create a user login, we will be able to create chats, invite other users to chats and we will update the list of messages in real time with WebSockets.

Development environment

Let's start by generating the base template:

npx create-next-app@latest chat-app --typescript --tailwind --app --eslint --use-npm

Taking into account the previous command we can see that a folder called chat-app/ will be created which will contain an initialized ESLint and Tailwindd CSS configuration, with TypeScript included and we will use NPM to install the dependencies (without forgetting to mention that the App Router will be used).

To enable rapid user interface iteration we will use the NextUI library, however step-by-step instructions on how to configure it will not be shown, so I recommend following this guide.

Database Schema

First, we will install the necessary dependencies:

npm install drizzle-orm better-sqlite3
npm install --dev drizzle-kit @types/better-sqlite3

Now we will define the schema of the database, which will consist of 4 tables. Starting with the users table, we will only have two columns which will be the id and the username and we must ensure that this is unique.

// db/schema.ts
import { relations } from "drizzle-orm";
import { sqliteTable, integer, text, unique } from "drizzle-orm/sqlite-core";

/**
 * Table Definitions
 */

export const users = sqliteTable("users", {
  id: integer("id").primaryKey({ autoIncrement: true }),
  username: text("username").unique().notNull(),
});

// ...

Next, we will define the conversations table, which will also have just two columns, namely the id and the name, the latter corresponding to the name of the chat/conversation.

// db/schema.ts

// ...

export const conversations = sqliteTable("conversations", {
  id: integer("id").primaryKey({ autoIncrement: true }),
  name: text("name").notNull(),
});

// ...

Moving now to defining the message table, we will define 4 columns, these being the id, body which corresponds to the content of the message and two foreign keys conversationId and senderId which we will later establish the relationship between the table of conversations and users.

// db/schema.ts

// ...

export const messages = sqliteTable("messages", {
  id: integer("id").primaryKey({ autoIncrement: true }),
  body: text("body").notNull(),
  conversationId: integer("conversation_id")
    .notNull()
    .references(() => conversations.id),
  senderId: integer("sender_id")
    .notNull()
    .references(() => users.id),
});

// ...

A super important table to define is the participants of each conversation, to which each row of the table will correspond to the record of an entry that must have a unique constraint between the conversationId and userId columns.

// db/schema.ts

// ...

export const participants = sqliteTable(
  "participants",
  {
    id: integer("id").primaryKey({ autoIncrement: true }),
    conversationId: integer("conversation_id")
      .notNull()
      .references(() => conversations.id),
    userId: integer("user_id")
      .notNull()
      .references(() => users.id),
  },
  (table) => ({
    participantUniqueConstraint: unique("participant_unique_constraint").on(
      table.conversationId,
      table.userId
    ),
  })
);

// ...

With the table schema defined, we can now formalize the definition of the relationships between the different tables.

// db/schema.ts

// ...

/**
 * Table Relationships
 */

export const userRelations = relations(users, ({ many }) => ({
  messages: many(messages),
  participants: many(participants),
}));

export const conversationRelations = relations(conversations, ({ many }) => ({
  messages: many(messages),
  participants: many(participants),
}));

export const messageRelations = relations(messages, ({ one }) => ({
  conversation: one(conversations, {
    fields: [messages.conversationId],
    references: [conversations.id],
  }),
  sender: one(users, {
    fields: [messages.senderId],
    references: [users.id],
  }),
}));

export const participantRelations = relations(participants, ({ one }) => ({
  conversation: one(conversations, {
    fields: [participants.conversationId],
    references: [conversations.id],
  }),
  user: one(users, {
    fields: [participants.userId],
    references: [users.id],
  }),
}));

With the database tables and columns defined, as well as the constraints and relationships between them, we can move on to the next step, which would be creating the connection with the database, as well as defining the database client with the inference of data types from the schemas we just defined in db/schema.ts.

// db/client.ts
import {
  drizzle,
  type BetterSQLite3Database,
} from "drizzle-orm/better-sqlite3";
import Database from "better-sqlite3";

import * as schema from "./schema";

const sqlite = new Database("local.db");

export const db: BetterSQLite3Database<typeof schema> = drizzle(sqlite, {
  schema,
});

Soketi Setup

In this step we will proceed to install Soketi on our equipment, for this we have two popular solutions. Either we install the CLI or run it in a Docker Container, I would recommend using Docker and for that I will share the following with you:

CLI installation Guide
Docker installation guide

After completing one of these guides and having Soketi working correctly on your machine, we can continue with this article and install the following dependencies in the project:

npm install pusher pusher-js

With the above dependencies installed, we can now create a folder called soketi/ with the following file:

// soketi/index.ts
import PusherServer from "pusher";
import PusherClient from "pusher-js";

export const pusherServer = new PusherServer({
  appId: "app-id",
  key: "app-key",
  secret: "app-secret",
  cluster: "",
  useTLS: false,
  host: "127.0.0.1",
  port: "6001",
});

export const pusherClient = new PusherClient("app-key", {
  cluster: "",
  httpHost: "127.0.0.1",
  httpPort: 6001,
  wsHost: "127.0.0.1",
  wsPort: 6001,
  wssPort: 6001,
  forceTLS: false,
  enabledTransports: ["ws", "wss"],
  authTransport: "ajax",
  authEndpoint: "/api/pusher-auth",
  auth: {
    headers: {
      "Content-Type": "application/json",
    },
  },
});

In the code above, two Pusher clients were created, one that will be used on the server side and another that will be used only on the client side.

The client configuration that will be used on the server side only has some basic settings such as address, port and some default values that are present in this documentation page.

While the one that will be used on the client side follows some of the definitions that were inspired from here with some changes regarding to authentication and authorization endpoints for the consumption of Pusher protocol channels.

Let's Build

API Routes

In today's article we will need to define an API Route to check whether the user is authorized to consume Pusher channels. I recommend reading this page of the documentation to have more detailed information on the subject as we will have a very simple authorization.

// app/api/pusher-auth/route.ts
import { pusherServer } from "@/soketi";

export async function POST(req: Request) {
  const data = await req.text();
  const [socketId, channelName] = data
    .split("&")
    .map((str) => str.split("=")[1]);

  const authResponse = pusherServer.authorizeChannel(socketId, channelName);

  return new Response(JSON.stringify(authResponse));
}

Login Page

To create today's Login page I decided to use the React Hook Form library to validate the form together with the valibot library to define the validation scheme through which we will enforce the data structure and data types at runtime of the form before submitting it.

The form will have only one controlled input that corresponds to the user's username and if, after submitting the form, it is not unique, an error message will be displayed.

// app/page.tsx
"use client";
import { useCallback } from "react";
import { Button, Input } from "@nextui-org/react";
import { minLength, object, string, Input as Infer } from "valibot";
import { Controller, SubmitHandler, useForm } from "react-hook-form";
import { valibotResolver } from "@hookform/resolvers/valibot";

import { addUserAction } from "@/services";

const schema = object({
  username: string([minLength(3)]),
});

export type FormValues = Infer<typeof schema>;

export default function Index() {
  const { handleSubmit, control, setError } = useForm<FormValues>({
    defaultValues: {
      username: "",
    },
    mode: "onSubmit",
    resolver: valibotResolver(schema),
  });

  const onSubmit: SubmitHandler<FormValues> = useCallback(
    async (data) => {
      try {
        await addUserAction(data);
      } catch (cause) {
        if (cause instanceof Error) {
          setError("username", { message: cause.message });
        }
      }
    },
    [setError]
  );

  return (
    <div className="h-screen w-screen flex items-center justify-center">
      <div className="flex flex-col items-center space-y-4 w-1/5">
        <Controller
          name="username"
          control={control}
          render={({ field, formState }) => (
            <Input
              {...field}
              label="Username"
              isInvalid={!!formState.errors.username?.message}
              errorMessage={formState.errors.username?.message}
            />
          )}
        />
        <Button
          type="button"
          onClick={handleSubmit(onSubmit)}
          color="primary"
          variant="shadow"
          fullWidth
        >
          Join
        </Button>
      </div>
    </div>
  );
}

Next we will need to create a folder called services/ which will contain the index.ts file which will hold a set of Server Functions. But for now let's just create the addUserAction.

// services/index.ts
"use server";
import { redirect } from "next/navigation";

import type { FormValues as IAddUser } from "@/app/page";
import { db } from "../db/client";
import { users } from "../db/schema";

export async function addUserAction(data: IAddUser) {
  let userId: number | undefined;

  const user = await db.query.users.findFirst({
    where: (user, { eq }) => eq(user.username, data.username),
  });

  userId = user?.id;

  if (!userId) {
    const result = await db.insert(users).values(data);
    const rowId = result.lastInsertRowid;
    if (rowId < 1 || typeof rowId !== "number") {
      throw Error("An error has occurred.");
    } else {
      userId = rowId;
    }
  }

  redirect(`/conversations/${userId}`);
}

// ...

Taking into account the previous function, we insert the new user into the database only if the username is unique and the insertion is successful, redirection is made to the conversation pages, otherwise an error is thrown.

Conversations Page

The next page that needs to be created is related to the Conversations. The first component that we can create will have the responsibility to create a new conversation, which in this case is just a button that invokes a Server Action, as follows:

// components/CreateConversation.tsx
"use client";
import { useCallback, type FC, type MouseEventHandler } from "react";
import { Button } from "@nextui-org/react";

import { bootstrapNewConversationAction } from "@/services";

interface Props {
  userId: number;
}

export const CreateConversation: FC<Props> = ({ userId }) => {
  const onClickHandler: MouseEventHandler<HTMLButtonElement> = useCallback(
    async (evt) => {
      evt.stopPropagation();
      try {
        await bootstrapNewConversationAction(userId);
      } catch (cause) {
        console.error(cause);
      }
    },
    [userId]
  );

  return (
    <Button
      type="button"
      color="primary"
      variant="flat"
      fullWidth
      onClick={onClickHandler}
    >
      New Conversation
    </Button>
  );
};

Taking into account the database schema we would need to give a unique name for the chat, to reduce complexity we will generate a unique identifier by installing the following dependency:

npm install nanoid

Now we need to create the bootstrapNewConversationAction function in the services/ folder which may look similar to the following:

// services/index.ts
"use server";
import { redirect } from "next/navigation";
import { nanoid } from "nanoid";

import type { FormValues as IAddUser } from "@/app/page";
import { db } from "../db/client";
import { conversations, participants, users } from "../db/schema";

// ...

export async function bootstrapNewConversationAction(userId: number) {
  const conversation = await db
    .insert(conversations)
    .values({ name: nanoid() });

  const conversationId = conversation.lastInsertRowid;

  if (typeof conversationId !== "number") {
    throw new Error("An error has occurred.");
  }

  const result = await db
    .insert(participants)
    .values({ userId, conversationId });

  if (result.changes < 1) {
    throw new Error("An error has occurred.");
  }

  redirect(`/conversations/${userId}/${conversationId}`);
}

This way we can now define the Layout of the page that will be responsible for rendering the <CreateConversation /> component and the remaining content of the route, including children (which would be the content of the pages).

However, we don't stop there because we will need to render a set of elements that correspond to a Conversation list in which the user is a participant.

// app/conversations/[userId]/layout.tsx
import type { PropsWithChildren } from "react";
import Link from "next/link";

import { CreateConversation } from "@/components/CreateConversation";
import { db } from "@/db/client";

interface Props {
  params: {
    userId: string;
  };
}

export default async function Layout({
  params,
  children,
}: PropsWithChildren<Props>) {
  const userId = Number(params.userId);

  if (!params || !params.userId || isNaN(userId)) {
    throw Error("The user identifier must be provided.");
  }

  const result = await db.query.users.findFirst({
    where: (user, { eq }) => eq(user.id, userId),
    with: {
      participants: {
        with: {
          conversation: true,
        },
      },
    },
  });

  return (
    <div className="h-screen w-screen flex">
      <div className="w-2/6 h-full flex flex-col items-center justify-center space-y-4 border-r-1.5">
        <div className="w-[90%]">
          <CreateConversation userId={userId} />
        </div>

        <div className="bg-white w-[90%] h-5/6 p-2 flex flex-col space-y-4">
          {result?.participants.map((item) => (
            <Link
              key={item.conversationId}
              href={`/conversations/${userId}/${item.conversationId}`}
              className="h-16 w-full flex items-center justify-start border-1.5"
            >
              <span className="mx-4 truncate">
                {item.conversation.name}
              </span>
            </Link>
          ))}
        </div>
      </div>
      <div className="w-4/6 h-full">{children}</div>
    </div>
  );
}

The next step is quite simple, because theoretically we need a root route for the conversations, which in this case would correspond to an idle state in which we have not yet selected any conversation. Which might look similar to this:

// app/conversations/[userId]/page.tsx
export default async function Page() {
  return (
    <div className="h-full w-full flex flex-col items-center justify-center space-y-1">
      <span className="text-lg leading-relaxed">Select a conversation</span>
      <small className="text-gray-400 leading-relaxed">
        or create a new one
      </small>
    </div>
  );
}

Conversation Details

The Conversations details page is the route that will show the list of messages and a set of actions taking into account the selected conversation. This one will be a nested route that will swap content with the root page that we defined previously and this will have a set of features.

Starting with the simplest, we will create the component that will enable sharing the conversation link so that other users can join.

// components/ConversationInviteAction.tsx
"use client";
import { Button } from "@nextui-org/react";

interface Props {
  chatId: number;
}

export function ConversationInviteAction({ chatId }: Props) {
  return (
    <Button
      color="primary"
      variant="flat"
      onClick={(evt) => {
        evt.stopPropagation();
        navigator.clipboard.writeText(`http://localhost:3000/invite/${chatId}`);
      }}
    >
      Invite Link
    </Button>
  );
}

In the component above, what we do is create just one button in which, when propagating the onClick event, we copy a URL to the clipboard with the unique identifier of the conversion.

The next component to be created is related to sending a new message to the conversation. This component will contain a controlled input and a button that will invoke a server action that will insert a new message into the database and trigger a Pusher event.

// components/ConversationTextField.tsx
"use client";
import { useCallback } from "react";
import { Button, Input } from "@nextui-org/react";
import { minLength, object, string, number, Input as Infer } from "valibot";
import { Controller, SubmitHandler, useForm } from "react-hook-form";
import { valibotResolver } from "@hookform/resolvers/valibot";

import { sendMessageAction } from "@/services";

const schema = object({
  body: string([minLength(1)]),
  chatId: number(),
  userId: number(),
});

export type TextFieldFormValues = Infer<typeof schema>;

type Props = Pick<TextFieldFormValues, "chatId" | "userId">;

export function ConversationTextField({ chatId, userId }: Props) {
  const { handleSubmit, control, setError, reset } =
    useForm<TextFieldFormValues>({
      defaultValues: {
        body: "",
        chatId: Number(chatId),
        userId: Number(userId),
      },
      mode: "onSubmit",
      resolver: valibotResolver(schema),
    });

  const onSubmit: SubmitHandler<TextFieldFormValues> = useCallback(
    async (data) => {
      try {
        await sendMessageAction(data);
        reset();
      } catch (cause) {
        if (cause instanceof Error) {
          setError("body", { message: cause.message });
        }
      }
    },
    [reset, setError]
  );

  return (
    <div className="flex flex-row items-center justify-center space-x-4 h-full w-full">
      <div className="w-4/6">
        <Controller
          name="body"
          control={control}
          render={({ field, formState }) => (
            <Input
              {...field}
              isInvalid={!!formState.errors.body?.message}
              errorMessage={formState.errors.body?.message}
              placeholder="Type your message..."
            />
          )}
        />
      </div>
      <Button
        type="button"
        onClick={handleSubmit(onSubmit)}
        color="primary"
        variant="shadow"
      >
        Send
      </Button>
    </div>
  );
}

Still on the previous component, we need to create a Server Action called sendMessageAction in the services/ folder, as follows:

// services/index.ts
"use server";
import { redirect } from "next/navigation";
import { nanoid } from "nanoid";

import type { FormValues as IAddUser } from "@/app/page";
import { db } from "../db/client";
import { conversations, messages, participants, users } from "../db/schema";
import { TextFieldFormValues } from "@/components/ConversationTextField";
import { pusherServer } from "@/soketi";

// ...

export async function sendMessageAction(data: TextFieldFormValues) {
  const result = await db
    .insert(messages)
    .values({
      conversationId: data.chatId,
      senderId: data.userId,
      body: data.body,
    })
    .returning();

  for await (const item of result) {
    await pusherServer.trigger(
      data.chatId.toString(),
      "evt::new-message",
      item
    );
  }
}

Since we can send a new message, we need to list all the messages we have in a conversation. One thing we need to take into account is that we need to list all messages before making the WebSocket connection with Soketi, so that each new message that is sent is added in real time incrementally.

To do this, we will first install the following dependencies:

npm install clsx

Then we create the following component:

// components/MessageList.tsx
"use client";
import { useEffect, useRef, useState } from "react";
import { clsx } from "clsx";

import { pusherClient } from "@/soketi";

type Message = {
  body: string;
  id: number;
  conversationId: number;
  senderId: number;
};

interface Props {
  initialMessages: Array<Message>;
  userId: number;
  chatId: number;
}

export function MessageList({ initialMessages, userId, chatId }: Props) {
  const lastMessageRef = useRef<HTMLDivElement>(null);
  const [items, setItems] = useState<Array<Message>>(() => [
    ...initialMessages,
  ]);

  useEffect(() => {
    lastMessageRef.current?.scrollIntoView({ behavior: "smooth" });
  }, [items]);

  useEffect(() => {
    const channel = pusherClient
      .subscribe(chatId.toString())
      .bind("evt::new-message", (datum: Message) =>
        setItems((state) => [...state, datum])
      );

    return () => {
      channel.unbind();
    };
  }, []);

  return (
    <>
      {items.length > 0 ? (
        <div className="h-full w-full flex flex-col overflow-y-auto">
          {items.map((item, index) => (
            <div
              key={item.id}
              className={clsx([
                "mt-1.5 flex",
                item.senderId === userId ? "justify-end" : "justify-start",
                items.length - 1 === index && "mb-3",
              ])}
            >
              <div
                className={clsx([
                  "max-w-md rounded-xl p-3",
                  item.senderId === userId
                    ? "bg-blue-500 text-white rounded-br-none mr-4"
                    : "bg-gray-300 text-gray-800 rounded-bl-none ml-4",
                ])}
              >
                {item.body}
              </div>
            </div>
          ))}
          <div ref={lastMessageRef} />
        </div>
      ) : (
        <div className="h-full w-full flex flex-col items-center justify-center space-y-1">
          <span className="text-lg leading-relaxed text-gray-500">
            Be the first one to send the first text
          </span>
          <small className="text-gray-400 leading-relaxed">or just wait</small>
        </div>
      )}
    </>
  );
}

Taking into account the code above, we receive in the component's props the initial messages that ideally would be those that we obtain from the database to have an initial hydration of the component. To avoid a mismatch, a shallow copy was made of them before being used as the initial state of useState.

We then use a useEffect together with the Pusher client to establish the WebSocket connection to Soketi, which will send each message in real time taking into account the chat identifier and event of the new messages we subscribe to.

Each message that is received on the client will be added to the list of messages we have in the component's local state. As soon as the component is unmounted, we close the connection with Soketi.

Now that we have all the necessary pieces we can proceed to create the Conversation details page.

// app/conversations/[userId]/[chatId]/page.tsx
import { ConversationInviteAction } from "@/components/ConversationInviteAction";
import { ConversationTextField } from "@/components/ConversationTextField";
import { MessageList } from "@/components/MessageList";
import { db } from "@/db/client";

interface Props {
  params: {
    userId: string;
    chatId: string;
  };
}

export default async function Page({ params }: Props) {
  const chatId = Number(params.chatId);
  const userId = Number(params.userId);

  if (!params || !params.userId || !params.chatId || isNaN(chatId)) {
    throw Error("The conversation and user identifiers must be provided.");
  }

  const result = await db.query.messages.findMany({
    where: (message, { eq }) => eq(message.conversationId, chatId),
  });

  return (
    <div className="h-full flex flex-col justify-end items-end">
      <div className="w-full h-24 flex items-center justify-end border-b-1.5">
        <div className="mx-4">
          <ConversationInviteAction chatId={chatId} />
        </div>
      </div>

      <MessageList initialMessages={result} userId={userId} chatId={chatId} />

      <div className="w-full h-24 border-t-1.5">
        <ConversationTextField chatId={chatId} userId={userId} />
      </div>
    </div>
  );
}

Invitation Page

Last but not least, the creation of the page that will validate whether a user is able to join a conversation through the link that is shared with them remains to be done.

On this page we will have a form whose only input corresponds to filling in the username, if this is already present in the conversation you will get an error. Otherwise, you will be redirected to the conversation you were invited to.

// app/invite/[chatId]/page.tsx
"use client";
import { useCallback } from "react";
import { Button, Input } from "@nextui-org/react";
import { minLength, object, string, number, Input as Infer } from "valibot";
import { Controller, SubmitHandler, useForm } from "react-hook-form";
import { valibotResolver } from "@hookform/resolvers/valibot";

import { joinConversationAction } from "@/services";

const schema = object({
  username: string([minLength(3)]),
  chatId: number(),
});

export type InvitationFormValues = Infer<typeof schema>;

interface Props {
  params: {
    chatId: string;
  };
}

export default function Index({ params }: Props) {
  const { handleSubmit, control, setError } = useForm<InvitationFormValues>({
    defaultValues: {
      username: "",
      chatId: Number(params.chatId),
    },
    mode: "onSubmit",
    resolver: valibotResolver(schema),
  });

  const onSubmit: SubmitHandler<InvitationFormValues> = useCallback(
    async (data) => {
      try {
        await joinConversationAction(data);
      } catch (cause) {
        if (cause instanceof Error) {
          setError("username", { message: cause.message });
        }
      }
    },
    [setError]
  );

  return (
    <div className="h-screen w-screen flex items-center justify-center">
      <div className="flex flex-col items-center space-y-4 w-1/5">
        <h3 className="text-lg leading-relaxed">Invitation link</h3>
        <Controller
          name="username"
          control={control}
          render={({ field, formState }) => (
            <Input
              {...field}
              label="Username"
              isInvalid={!!formState.errors.username?.message}
              errorMessage={formState.errors.username?.message}
            />
          )}
        />
        <Button
          type="button"
          onClick={handleSubmit(onSubmit)}
          color="primary"
          variant="shadow"
          fullWidth
        >
          Join
        </Button>
      </div>
    </div>
  );
}

As usual, we need to create the Server Action joinConversationAction in the services/ folder.

// services/index.ts
"use server";
import { redirect } from "next/navigation";
import { nanoid } from "nanoid";

import type { FormValues as IAddUser } from "@/app/page";
import { db } from "../db/client";
import { conversations, messages, participants, users } from "../db/schema";
import type { InvitationFormValues } from "@/app/invite/[chatId]/page";
import { TextFieldFormValues } from "@/components/ConversationTextField";
import { pusherServer } from "@/soketi";

// ...

export async function joinConversationAction(data: InvitationFormValues) {
  const user = await db.query.users.findFirst({
    where: (user, { eq }) => eq(user.username, data.username),
  });

  const userId = user?.id;

  if (typeof userId !== "number") {
    throw new Error("An error has occurred.");
  }

  const result = await db
    .insert(participants)
    .values({ userId, conversationId: data.chatId })
    .onConflictDoNothing();

  if (result.changes < 1) {
    throw new Error("This username is already part of the chat");
  }

  redirect(`/conversations/${userId}/${data.chatId}`);
}

Conclusion

I hope you found this article helpful, whether you're using the information in an existing project or just giving it a try for fun.

Please let me know if you notice any mistakes in the article by leaving a comment. And, if you'd like to see the source code for this article, you can find it on the github repository linked below.

Github Repo