DEV Community: Delightful Engineering

React Developer Resources

Delightful Engineering — Sat, 07 Dec 2024 23:18:51 +0000

These days it probably doesn't even need to be mentioned anymore but obviously AI chat tools like ChatGPT, Claude and others are game changers for productivity and solving problems.

However, these resources are human created and are hopefully helpful. This doc will continue to be updated with new resources on our blog.

Neat Docs

Official React Tutorials

The official React docs are a great place to start. They are well written and cover a lot of ground. The learn section in particular is an essential starting point.

Composition vs Inheritance

This small page from the legacy React docs is still a gem. Starting to think in terms of composition really helps as you're growing a project and creating re-usable components.

State Management

Probably the best client-server state management library out there. Can be used for any sort of async work. If you find yourself spending a lot of time thinking about and optimizing loading and error states in your applications it is worth
considering React Query.

TkDodo has an written a ton of detailed advice and insight on how to really get the most out of it. See Practical React Query.

XState

State management libraries like React Query, RTK Query, and SWR are great for client-server state management. For complex client state application, XState is very good.

XState is a state management and orchestration solution for JavaScript and TypeScript apps.
It uses event-driven programming, state machines, statecharts, and the actor model to handle complex logic in predictable, robust, and visual ways. XState provides a powerful and flexible way to manage application and workflow state by allowing developers to model logic as actors and state machines.

The state charts in particular make XState unique. The ability to visualize your state machines is essentially a form interactive self-documentation.

Admin

React Admin

React Admin is a frontend framework for building admin applications. It's a great way to quickly build admin interfaces for your applications. It's built on top of Material UI and is very customizable.

For data management purposes it may be a waste of time to build a custom admin interface. CRUD operations are pretty standard and React Admin is a great way to get up and running quickly. Spend time on the parts of your application that are unique and provide value to users.

Debugging

React Dev Tools

Pretty standard dev tool most people familiar with React will have used this. In many cases those people just coming into the React ecosystem may not be aware how helpful this chrome extension can be.

React Profiler

The Profiler extension comes with React Dev tools and is a great way to look at the performance of component trees. The extension is just a UI on top of the React.Profiler API. You can also import the Profiler component from react, wrap it around any component tree and get the same data if you wanted to log it or do something with the data.

With the extension you just press record and then interact with your app.

As an app grows in complexity you might start experiencing performance issues. It's really helpful to be able to see which components are causing the most re-renders and how long they are taking.

This Profiler image is from a recording of this site. The site is almost completely static and rendered on the server, so the React Profiler is pretty boring. We can see that the CustomLink component (The Delightful Engineering logo in the top left) is re-rendering very quickly.

Testing

A lot of people tend to follow the testing pyramid. The idea is that you should have a lot of unit tests, some integration tests, and a few end-to-end tests.

The idea is that unit tests run the fastest and are the cheapest to write. They should make up the bulk of your tests. Integration tests are more expensive to write and run, but they test how different parts of your application work together. End-to-end tests are the most expensive to write and run, but they test how your application works from the user's perspective.

Unit tests can tend towards becoming bloated and not very useful. It's important to keep them focused on the smallest possible unit of code.

The only thing worse than no tests is a glob of tests that are not useful.

Unit & Integration

Jest

Jest is the most popular testing library for Javascript. It's built by Facebook and is the default testing library for React.

React Testing Library

For testing hooks and components, React Testing Library is the way to go.

End to End

Cypress

Cypress is a very mature end-to-end testing framework.

Playwright

Playwright is a newer end-to-end testing framework that is built by Microsoft. It's similar to Cypress but has some unique features.

Blogs, Videos and People

Building Resilient Frontend Architecture - Monica Lent

This talk by Monica Lent in 2019 is a timeless masterpiece for frontend developers.

Among many other interesting things, she suggests "shared components" should either be in your design system or should be copy-pasted into the locations where they are meant to be used.

Matt Pocock

Matt Pocock is one of the best educators and content creators focusing heavily on Typescript. He has put out loads of Typescript stuff on his socials and offers a Total Typescript educational package that will allegedly turn you into a wizard.

Overreacted

One of the long time core maintainers of React Dan Abromov's blog overreacted is fun to read. Some interesting pieces are What are the React Team Principles and Writing Resilient Components.

Kent Dodd's Blog

Kent has written some of the best content for web developers on the internet. Among other things he's created several comprehensive course Epic Web, Epic React and Testing Javascript. He's also the creator of React Testing Library.

Drawing on the Web with Canvas Rendering Context

Delightful Engineering — Fri, 29 Nov 2024 16:46:17 +0000

How would you go about creating a 2D drawing app in the browser?

The html canvas element in combination with the CanvasRenderingContext2D interface provides a simple way to draw graphics on the web.

Note: Two alternative options for drawing on the web are WebGL and SVG.

CanvasRenderingContext2D

The CanvasRenderingContext2D interface provides the 2D rendering context for the drawing surface of a canvas element. It is used to draw shapes, text, images, and other objects on the canvas.

This is raster graphics as opposed to vector graphics which is what SVG uses. Raster graphics are made up of pixels, while vector graphics are made up of paths. Raster does not
scale up and down as well as vector graphics.

Use Cases

Some common use cases for the 2D canvas include:

Photo editing
Drawing / sketching apps
Simple games

Example App

In this example we have a couple of simple requirements:

User can draw lines on the canvas
User can set a background
User can choose background and line colors from an EyeDropper color picker and default provided color pallette
User can clear the canvas
User can undo / redo
User can download the drawing as an image
User can do all of the above with hotkeys

codesandbox.io

Feature Ideas

You could take this example further by adding more features such as:

Pen customization: a form to allow the user to customize the size and brush stroke of the pen.

Saving image state to a persistent store: the image state is currently stored in component state. You could save the image state to a persistent store such as local storage or a database.

Adding text: add a new action button to allow the user to add text to the canvas. The CanvasRenderingContext2D interface provides
strokeText and fillText methods for drawing text on the canvas.

Adding images: allow the user to upload images to the canvas. The CanvasRenderingContext2D interface provides an drawImage method for drawing images on the canvas. This could really open up many possibilities for the app.

Adding a layer system: allow the user to add multiple layers to the canvas. This would allow the user to draw on different layers and toggle the visibility of each layer. This would be a more complex feature to implement but would provide a lot of flexibility to the user.

Fill area: add a new action button to allow the user to fill an area with a color. The CanvasRenderingContext2D interface provides a fill method for filling an area with the current fill style.

Final Thoughts

Its worth mentioning again that WebGL and SVG are two other interesting options for drawing on the web that are worth looking into.

The MDN docs for the CanvasRenderingContext2D interface are a great resource for learning more about the 2D canvas.

Take this example and run with it! There are so many possibilities for creating interesting and interactive drawing apps on the web.

How To Learn Programming

Delightful Engineering — Sat, 22 Jul 2023 22:12:05 +0000

You’re never going to learn programming by watching a video called ‘learn programming’. The only way to learn programming, and everyone who I’ve ever met who can program well learned it all in the same way. They had something they wanted to do, and then they tried to do it. Then they were like, ‘oh it would be nice if the computer could do this thing’, and that’s how you learn, you just keep pushing on a project. The only advice I have for learning programming is - go program. - geohotz

Start with a problem

Here’s a problem.

I struggle with procrastination. When I force myself to ‘just start’, after the first 10 minutes, all of the sudden I’m in a flow state and the procrastination is gone. How can I use a computer to help me overcome this problem?

Solve the problem with minimum functionality

Commit yourself to solving this problem.

Here’s a potential solution to that problem.

Build yourself a mobile app. There are many ways to build mobile apps - part of this is just finding one framework to do this with. A google search will give you many different platforms / frameworks / languages by which to do this. Just pick one - React Native for example.

Have the app send you a push notification every morning reminding you that procrastination is just your limbic system playing games with you. You have a prefrontal cortex, you’re better than that. You can self-reflect. Go get a yerba mate and start for 10 minutes.

Evaluate the error rate of your solution

Solving solutions with computers usually involves creating some kind of feedback loop. Have the app send you a notification at the end of the day which could be like a form that you fill out that has things like, rate yourself from 1-10 on how well you did on overcoming procrastination. Maybe there’s a text box where you write what problems you solved that day. Store these things in a database. A google search will give you a bunch of database options - Supabase for example.

Now you have access to a history of self-evaluation that you can evaluate how well your solution is working.

Make the program better

Now that you have a feedback loop, you can start making the program better. If you’re regressing over time, probably a new feature that tells you how bad you’ve failed to overcome procrastination for the past week would help you improve.

You now also have a history of all the small things you’ve accomplished overtime. Do you remember all the small wins you have over the past 20 years? No. This data is valuable, you could probably do something special with it.

Get users

Allow other users to create their own accounts through an auth provider like (sign in with your google account) or something like it.

That database you have would now need to support storing any user’s data. Things get interesting now but this is just another problem to solve.

If you’re lost, do this first

Make a codecademy account.

Pick any language and go through the course.

Understand:

Variables
Data types
- Strings
- Booleans
- Integers
- Floats
- Arrays
- Etc…
Conditional statements
- If, elseif, else, etc.
- Switch statements
Functions
Classes
Loops
Control flow (the order in which a program runs)

By understanding these basic concepts, they will transfer to all languages. After learning these concepts, learning new languages becomes easier and easier.

After this you could go through a more general course like “web development” or “data science”. Maybe even the “iOS Developer” one would help getting started on solving the procrastination problem.

Eventually do the “Computer Science” course. Then watch a harvard / mit free online CS 50/101 courses. Learning the fundamentals of computer science will make you a deadly weapon. Not required to start though.

Understanding basic math, linear algebra and discrete math will take you a long way.

Last thing

This is a marathon, not a sprint. Ultimately this is no different from exercising, the more you put in the more you get out. The more things you build, the more credit you have - the more proof you have that you can convince someone else to give you money to solve problems.

Oh, and learn Git. Make a Github account so that you can keep all of your programs there. Git will let you control and version your program - and it allows other people to work on your program. Learning Git is essential.

Codecademy has a good Git & Github course.

In general, spend the minimum possible time on courses / tutorials that you can. Banging your head against solving a problem will get you astronomically further.

A good video to watch is George Hotz | Programming | what is programming? (noob lessons!) | Science & Technology. Try to take what he says for what it is, and don't anything personal.

Nest JS Websockets - Rate Limiting and Acknowledgements

Delightful Engineering — Wed, 28 Dec 2022 00:31:50 +0000

Welcome to Part 5: Rate Limiting, of building a realtime chat application with Nest JS and React.

In this part of the series, we’re going to implement rate limiting! With rate limiting, we will likely be denying users from sending messages - so to be respectful we should indicate to the user whether or not their message was delivered. To achieve this, we will also be introducing a new mechanism for our chat events called acknowledgements in socket.io parlance.

If you haven’t checkout out previous parts of this series, go back to follow along.

Part 1: Basics - laying the foundations of the application and implementing the most basic websockets features.
Part 2: Rooms - implementing websocket rooms and major UI improvements.
Part 3: Pipes and E2E Validation with Zod - validate mutations with Zod schemas shared between client and server. From form actions, to emitting client events, to handling events on the server - giving our application multiple levels of protection from malformed data.
Part 4: Guards and Authorization with CASL - attribute-based authorization with CASL and guards.

This series will attempt to touch on as many features of Nest's websockets integration as well as socket.io in general.

The entire series will be building on one repository that has both the server and the client for the application, so for each part you can download the repository from specific commits to follow along or you can clone the entire repository in it's finished state.

Download the repository at this commit to follow along - we’re not going to cover everything to keep the focus on rate limiting and acknowledgements.

Rate Limiting

Rate limiting is a process by which we can limit a user or agent from performing an excessive amount of actions in a given time frame. This is a critical part of developing any kind of application to prevent abusive use of your server or database and is one way to mitigate basic DDoS attacks.

Throttler Guard

Nest provides easy-to-use support for rate limiting with websockets. We will implement a WsThrottlerGuard by extending ThrottlerGuard from @nestjs/throttler.

In our guard’s handleRequest method, we have access to the execution context from which we can get a reference to the client’s IP address. We then use built-in methods from ThrottlerGuard to store the amount of times a client has sent a websocket event and we compare that with a ttl (time-to-live) that is passed to the guard.

src/server/chat/guards/throttler.guard.ts

import { Injectable, ExecutionContext } from '@nestjs/common'
import { ThrottlerGuard, ThrottlerException } from '@nestjs/throttler'

@Injectable()
export class WsThrottlerGuard extends ThrottlerGuard {
  async handleRequest(context: ExecutionContext, limit: number, ttl: number): Promise<boolean> {
    const client = context.switchToWs().getClient()
    const ip = client.conn.remoteAddress
    const key = this.generateKey(context, ip)
    const ttls = await this.storageService.getRecord(key)

    if (ttls.length >= limit) {
      throw new ThrottlerException()
    }

    await this.storageService.addRecord(key, ttl)
    return true
  }
}

We also need to include the ThrottlerModule from @nestjs/throttler in the module that will use it - in our case the chat module.

We can pass a default limit and ttl to it’s forRoot method. This means any gateway handler that implements our guard will by default allow 5 events in a 60 second time span.

import { Module } from '@nestjs/common'
import { ChatGateway } from './chat.gateway'
import { RoomModule } from '../room/room.module'
import { UserModule } from '../user/user.module'
import { CaslModule } from '../casl/casl.module'
import { ThrottlerModule } from '@nestjs/throttler'

@Module({
  imports: [RoomModule, UserModule, CaslModule, ThrottlerModule.forRoot({ limit: 5, ttl: 60 })],
  providers: [ChatGateway],
})
export class ChatModule {}

Throttler Guard Usage in Websocket Gateway

The only thing we now need to do is to use our guards in our chat websocket gateway via the @UseGuards decorator. If you’ve landed on this part of the series and want context for everything happening in the gateway, you can go back an skim through previous parts - but you don’t necessarily need to.

src/server/chat/chat.gateway.ts

import {
  MessageBody,
  SubscribeMessage,
  WebSocketGateway,
  WebSocketServer,
  OnGatewayConnection,
  OnGatewayDisconnect,
} from '@nestjs/websockets'
import { Logger, UseGuards, UsePipes } from '@nestjs/common'
import {
  ServerToClientEvents,
  ClientToServerEvents,
  Message,
  JoinRoom,
  KickUser,
} from '../../shared/interfaces/chat.interface'
import { Server, Socket } from 'socket.io'
import { RoomService } from '../room/room.service'
import { ZodValidationPipe } from '../pipes/zod.pipe'
import { ChatMessageSchema, JoinRoomSchema, KickUserSchema } from '../../shared/schemas/chat.schema'
import { UserService } from '../user/user.service'
import { ChatPoliciesGuard } from './guards/chat.guard'
import { WsThrottlerGuard } from './guards/throttler.guard'
import { Throttle } from '@nestjs/throttler'

@WebSocketGateway({})
export class ChatGateway implements OnGatewayConnection, OnGatewayDisconnect {
  constructor(private roomService: RoomService, private userService: UserService) {}

  @WebSocketServer() server: Server = new Server<ServerToClientEvents, ClientToServerEvents>()

  private logger = new Logger('ChatGateway')

  @Throttle(10, 30)
  @UseGuards(ChatPoliciesGuard<Message>, WsThrottlerGuard)
  @UsePipes(new ZodValidationPipe(ChatMessageSchema))
  @SubscribeMessage('chat')
  async handleChatEvent(
    @MessageBody()
    payload: Message
  ): Promise<boolean> {
    this.logger.log(payload)
    this.server.to(payload.roomName).emit('chat', payload)
    return true
  }

  @UseGuards(ChatPoliciesGuard<JoinRoom>, WsThrottlerGuard)
  @UsePipes(new ZodValidationPipe(JoinRoomSchema))
  @SubscribeMessage('join_room')
  async handleSetClientDataEvent(
    @MessageBody()
    payload: JoinRoom
  ): Promise<boolean> {
    this.logger.log(`${payload.user.socketId} is joining ${payload.roomName}`)
    await this.userService.addUser(payload.user)
    await this.server.in(payload.user.socketId).socketsJoin(payload.roomName)
    await this.roomService.addUserToRoom(payload.roomName, payload.user.userId)
    return true
  }

  @UseGuards(ChatPoliciesGuard<KickUser>)
  @UsePipes(new ZodValidationPipe(KickUserSchema))
  @SubscribeMessage('kick_user')
  async handleKickUserEvent(@MessageBody() payload: KickUser): Promise<boolean> {
    this.logger.log(`${payload.userToKick.userName} is getting kicked from ${payload.roomName}`)
    await this.server.to(payload.roomName).emit('kick_user', payload)
    await this.server.in(payload.userToKick.socketId).socketsLeave(payload.roomName)
    await this.server.to(payload.roomName).emit('chat', {
      user: {
        userId: 'serverId',
        userName: 'TheServer',
        socketId: 'ServerSocketId',
      },
      timeSent: new Date(Date.now()).toLocaleString('en-US'),
      message: `${payload.userToKick.userName} was kicked.`,
      roomName: payload.roomName,
    })
    return true
  }

  async handleConnection(socket: Socket): Promise<void> {
    this.logger.log(`Socket connected: ${socket.id}`)
  }

  async handleDisconnect(socket: Socket): Promise<void> {
    const user = await this.roomService.getFirstInstanceOfUser(socket.id)
    if (user !== 'Not Exists') {
      await this.userService.removeUserById(user.userId)
    }
    await this.roomService.removeUserFromAllRooms(socket.id)
    this.logger.log(`Socket disconnected: ${socket.id}`)
  }
}

Let’s drill into the important parts.

We have a handler that is subscribe to 'chat' events that handles messages. The default limit of 5 events per 60 seconds that we passed in the ThrottlerModule's forRoot method isn’t exactly what we want for chat messages. We want to increase this limit - which we can customize per handler by using the @Throttle(10, 30) decorator to increase the limit to 10 events per 30 seconds ttl.

@Throttle(10, 30)
@UseGuards(ChatPoliciesGuard<Message>, WsThrottlerGuard)
@UsePipes(new ZodValidationPipe(ChatMessageSchema))
@SubscribeMessage('chat')
async handleChatEvent(
  @MessageBody()
  payload: Message,
): Promise<boolean> {
  //...
}

The second handler we want to throttle is the one subscribed to 'join_room' events. Since we didn’t pass @Throttle, this will default to 5 actions per 60 seconds.

@UseGuards(ChatPoliciesGuard<JoinRoom>, WsThrottlerGuard)
@UsePipes(new ZodValidationPipe(JoinRoomSchema))
@SubscribeMessage('join_room')
async handleSetClientDataEvent(
  @MessageBody()
  payload: JoinRoom,
): Promise<boolean> {
  /...
}

In the event that one of our throttler guards detect a rate limit hit - an exception is thrown! The request will not go any further.

[1] [Nest] 53620  - 12/27/2022, 11:09:15 AM   ERROR [WsExceptionsHandler] ThrottlerException: Too Many Requests
[1] ThrottlerException: ThrottlerException: Too Many Requests
[1]     at WsThrottlerGuard.<anonymous> (/Users/austinhoward/code/nest-realtime/nest-react-websockets/src/server/chat/guards/throttler.guard.ts:17:13)
[1]     at Generator.next (<anonymous>)
[1]     at fulfilled (/Users/austinhoward/code/nest-realtime/nest-react-websockets/dist/server/server/chat/guards/throttler.guard.js:11:58)
[1]     at processTicksAndRejections (internal/process/task_queues.js:93:5)

Beyond brute force attacks, imagine we also made database calls or requests to external services in our gateway handler functions. We are now also protecting upstream services.

Acknowledgements

Now that we’ve implemented rate limiting, and also from previous parts of the series we’ve added data validation pipes and authorization guards - we need to start indicating back to the client whether or not their event was successful.

An important part of a good user experience is letting the user know what is happening - especially if there are failure cases!

For web socket based applications this is where acknowledgements come in handy.

Schemas

The first thing we need to do is update our zod schemas. Our web socket gateway event handlers should now accept an additional argument which is a callback function.

Here is the entire file with all of our schemas.

src/shared/schemas/chat.schema.ts

import { z } from 'zod'

export const UserIdSchema = z.string().min(1).max(24)

export const UserNameSchema = z
  .string()
  .min(1, { message: 'Must be at least 1 character.' })
  .max(16, { message: 'Must be at most 16 characters.' })

export const MessageSchema = z
  .string()
  .min(1, { message: 'Must be at least 1 character.' })
  .max(1000, { message: 'Must be at most 1000 characters.' })

export const TimeSentSchema = z.number()

export const RoomNameSchemaRegex = new RegExp('^\\S+\\w$')

export const RoomNameSchema = z
  .string()
  .min(2, { message: 'Must be at least 2 characters.' })
  .max(16, { message: 'Must be at most 16 characters.' })
  .regex(RoomNameSchemaRegex, {
    message: 'Must not contain spaces or special characters.',
  })

export const EventNameSchema = z.enum(['chat', 'kick_user', 'join_room'])

export const SocketIdSchema = z.string().length(20, { message: 'Must be 20 characters.' })

export const UserSchema = z.object({
  userId: UserIdSchema,
  userName: UserNameSchema,
  socketId: SocketIdSchema,
})

export const ChatMessageSchema = z.object({
  user: UserSchema,
  timeSent: TimeSentSchema,
  message: MessageSchema,
  roomName: RoomNameSchema,
  eventName: EventNameSchema,
})

export const RoomSchema = z.object({
  name: RoomNameSchema,
  host: UserSchema,
  users: UserSchema.array(),
})

export const JoinRoomSchema = z.object({
  user: UserSchema,
  roomName: RoomNameSchema,
  eventName: EventNameSchema,
})

export const KickUserSchema = z.object({
  user: UserSchema,
  userToKick: UserSchema,
  roomName: RoomNameSchema,
  eventName: EventNameSchema,
})

export const KickUserEventAckSchema = z.function().args(z.boolean()).returns(z.void())

export const KickUserEventSchema = z
  .function()
  .args(KickUserSchema, KickUserEventAckSchema)
  .returns(z.void())

export const ChatEventAckSchema = z.function().args(z.boolean()).returns(z.void())

export const ChatEventSchema = z
  .function()
  .args(ChatMessageSchema, ChatEventAckSchema)
  .returns(z.void())

export const JoinRoomEventAckSchema = z.function().args(z.string(), z.boolean()).returns(z.void())

export const JoinRoomEventSchema = z
  .function()
  .args(JoinRoomSchema, JoinRoomEventAckSchema)
  .returns(z.void())

export const ClientToServerEventsSchema = z.object({
  chat: ChatEventSchema,
  join_room: JoinRoomEventSchema,
  kick_user: KickUserEventSchema,
})

export const ServerToClientEventsSchema = z.object({
  chat: ChatEventSchema,
  kick_user: KickUserEventSchema,
})

Let’s focus in on the new acknowledgement callbacks.

With zod we can define schemas for everything including functions. Our event handler handles kicking users. We simply want the server to return a boolean indicating whether or not the handler successfully executed and did in fact kick the user.

// Kick user event callback function schema
export const KickUserEventAckSchema = z.function().args(z.boolean()).returns(z.void())

// Kick user event handler schema
export const KickUserEventSchema = z
  .function()
  .args(KickUserSchema, KickUserEventAckSchema)
  .returns(z.void())

Our chat event for handling chat messages we do the exact same schema for it’s callback.

// Chat event callback function schema
export const ChatEventAckSchema = z.function().args(z.boolean()).returns(z.void())

// Chat event handler schema
export const ChatEventSchema = z
  .function()
  .args(ChatMessageSchema, ChatEventAckSchema)
  .returns(z.void())

For the join room event, we’re going to do something similar but we’re going to use an additional argument in the callback function for a error message. The first argument in the JoinRoomEventAckSchema will be the error message (z.string) and the second one is the boolean returned from the handler function on the server just like the previous two events.

export const JoinRoomEventAckSchema = z.function().args(z.string(), z.boolean()).returns(z.void())

export const JoinRoomEventSchema = z
  .function()
  .args(JoinRoomSchema, JoinRoomEventAckSchema)
  .returns(z.void())

Acknowledgements Usage in Websocket Gateway

In the previous section where we added the ThrottlerGuard for rate limiting we saw the updated gateway. To support the new callback schemas we’re defined for our events, we simply need to return true from our handler functions.

src/server/chat/chat.gateway.ts

  @Throttle(10, 30)
  @UseGuards(ChatPoliciesGuard<Message>, WsThrottlerGuard)
  @UsePipes(new ZodValidationPipe(ChatMessageSchema))
  @SubscribeMessage('chat')
  async handleChatEvent(
    @MessageBody()
    payload: Message,
  ): Promise<boolean> {
    this.logger.log(payload);
    this.server.to(payload.roomName).emit('chat', payload);
    return true; // <---- return true when the handler function finishes
  }

  @UseGuards(ChatPoliciesGuard<JoinRoom>, WsThrottlerGuard)
  @UsePipes(new ZodValidationPipe(JoinRoomSchema))
  @SubscribeMessage('join_room')
  async handleSetClientDataEvent(
    @MessageBody()
    payload: JoinRoom,
  ): Promise<boolean> {
    this.logger.log(`${payload.user.socketId} is joining ${payload.roomName}`);
    await this.userService.addUser(payload.user);
    await this.server.in(payload.user.socketId).socketsJoin(payload.roomName);
    await this.roomService.addUserToRoom(payload.roomName, payload.user.userId);
    return true; // <---- return true when the handler function finishes
  }

  @UseGuards(ChatPoliciesGuard<KickUser>)
  @UsePipes(new ZodValidationPipe(KickUserSchema))
  @SubscribeMessage('kick_user')
  async handleKickUserEvent(
    @MessageBody() payload: KickUser,
  ): Promise<boolean> {
    this.logger.log(
      `${payload.userToKick.userName} is getting kicked from ${payload.roomName}`,
    );
    await this.server.to(payload.roomName).emit('kick_user', payload);
    await this.server
      .in(payload.userToKick.socketId)
      .socketsLeave(payload.roomName);
    await this.server.to(payload.roomName).emit('chat', {
      user: {
        userId: 'serverId',
        userName: 'TheServer',
        socketId: 'ServerSocketId',
      },
      timeSent: new Date(Date.now()).toLocaleString('en-US'),
      message: `${payload.userToKick.userName} was kicked.`,
      roomName: payload.roomName,
    });
    return true; // <---- return true when the handler function finishes
  }

Acknowledgements Usage in The Client

Now that we’ve added returns to our chat gateway handlers, we can now react to those on the client in a user friendly way. If you’ve been following along through the series, you’ll be familiar with how the client works up to this point. Here is our chat page that houses most of the application’s functionality.

src/client/pages/chat.tsx

import React, { useState, useEffect } from 'react'
import { MakeGenerics, useMatch, useNavigate } from '@tanstack/react-location'
import { io, Socket } from 'socket.io-client'
import {
  User,
  Message,
  ServerToClientEvents,
  ClientToServerEvents,
  KickUser,
  JoinRoom,
} from '../../shared/interfaces/chat.interface'
import { Header } from '../components/header'
import { UserList } from '../components/list'
import { MessageForm } from '../components/message.form'
import { Messages, ClientMessage } from '../components/messages'
import { ChatLayout } from '../layouts/chat.layout'
import { unsetRoom, useRoomQuery } from '../lib/room'
import { getUser } from '../lib/user'
import { ChatMessageSchema, JoinRoomSchema, KickUserSchema } from '../../shared/schemas/chat.schema'
import { LoadingLayout } from '../layouts/loading.layout'
import { Loading } from '../components/loading'

const socket: Socket<ServerToClientEvents, ClientToServerEvents> = io({
  autoConnect: false,
})

function Chat() {
  const {
    data: { user, roomName },
  } = useMatch<ChatLocationGenerics>()

  const [isConnected, setIsConnected] = useState(socket.connected)
  const [isJoinedRoom, setIsJoinedRoom] = useState(false)
  const [isJoiningDelay, setIsJoiningDelay] = useState(false)
  const [messages, setMessages] = useState<ClientMessage[]>([])
  const [toggleUserList, setToggleUserList] = useState<boolean>(false)

  const { data: room, refetch: roomRefetch } = useRoomQuery(roomName, isConnected)

  const navigate = useNavigate()

  useEffect(() => {
    if (!user || !roomName) {
      navigate({ to: '/', replace: true })
    } else {
      socket.on('connect', () => {
        setIsJoiningDelay(true)
        const joinRoom: JoinRoom = {
          roomName,
          user: { socketId: socket.id, ...user },
          eventName: 'join_room',
        }
        JoinRoomSchema.parse(joinRoom)
        setTimeout(() => {
          // default required 800 ms minimum join delay to prevent flickering
          setIsJoiningDelay(false)
        }, 800)
        socket.timeout(30000).emit('join_room', joinRoom, (err, response) => {
          if (err) {
            leaveRoom()
          }
          if (response) {
            setIsJoinedRoom(true)
          }
        })
        setIsConnected(true)
      })

      socket.on('disconnect', () => {
        setIsConnected(false)
      })

      socket.on('chat', (e) => {
        if (e.user.userId !== user.userId) {
          setMessages((messages) => [{ ...e, delivered: true }, ...messages])
        }
      })

      socket.on('kick_user', (e) => {
        if (e.userToKick.socketId === socket.id) {
          leaveRoom()
        }
      })

      socket.connect()
    }
    return () => {
      socket.off('connect')
      socket.off('disconnect')
      socket.off('chat')
      socket.off('kick_user')
    }
  }, [])

  const leaveRoom = () => {
    socket.disconnect()
    unsetRoom()
    navigate({ to: '/', replace: true })
  }

  const sendMessage = (message: string) => {
    if (user && socket && roomName) {
      const chatMessage: Message = {
        user: {
          userId: user.userId,
          userName: user.userName,
          socketId: socket.id,
        },
        timeSent: Date.now(),
        message,
        roomName: roomName,
        eventName: 'chat',
      }
      ChatMessageSchema.parse(chatMessage)
      setMessages((messages) => [{ ...chatMessage, delivered: false }, ...messages])
      socket.emit('chat', chatMessage, (response) => {
        if (response) {
          setMessages((messages) => {
            const previousMessageIndex = messages.findIndex((mes) => {
              if (mes.user.userId === user.userId && mes.timeSent === chatMessage.timeSent) {
                return mes
              }
            })
            if (previousMessageIndex === -1) {
              throw 'Previously sent message not found to update delivered status'
            }
            messages[previousMessageIndex] = {
              ...messages[previousMessageIndex],
              delivered: true,
            }
            return [...messages]
          })
        }
      })
    }
  }

  const kickUser = (userToKick: User) => {
    if (!room) {
      throw 'No room'
    }
    if (!user) {
      throw 'No current user'
    }
    const kickUserData: KickUser = {
      user: { ...user, socketId: socket.id },
      userToKick: userToKick,
      roomName: room.name,
      eventName: 'kick_user',
    }
    KickUserSchema.parse(kickUserData)
    socket.emit('kick_user', kickUserData, (response) => {
      if (response) {
        roomRefetch()
      }
    })
  }

  return (
    <>
      {user?.userId && roomName && room && isJoinedRoom && !isJoiningDelay ? (
        <ChatLayout>
          <Header
            isConnected={isConnected}
            users={room?.users ?? []}
            roomName={roomName}
            handleUsersClick={() => setToggleUserList((toggleUserList) => !toggleUserList)}
            handleLeaveRoom={() => leaveRoom()}
          ></Header>
          {toggleUserList ? (
            <UserList
              room={room}
              currentUser={{ socketId: socket.id, ...user }}
              kickHandler={kickUser}
            ></UserList>
          ) : (
            <Messages user={user} messages={messages}></Messages>
          )}
          <MessageForm sendMessage={sendMessage}></MessageForm>
        </ChatLayout>
      ) : (
        <LoadingLayout>
          <Loading message={`Joining ${roomName}`}></Loading>
        </LoadingLayout>
      )}
    </>
  )
}

export const loader = async () => {
  const user = getUser()
  return {
    user: user,
    roomName: sessionStorage.getItem('room'),
  }
}

type ChatLocationGenerics = MakeGenerics<{
  LoaderData: {
    user: Pick<User, 'userId' | 'userName'>
    roomName: string
  }
}>

export default Chat

When a client socket connects and is attempting to join a room, we want to improve the user experience by showing a loading screen until the server returns an acknowledgement from the 'join_room' event. To avoid flickering due to a fast connection, we also want to add a minimum delay of 800 milliseconds. To support this we need two useState hooks for when the user has successfully joined a room, and when the minimum 800 millisecond delay is finished.

const [isJoinedRoom, setIsJoinedRoom] = useState(false)
const [isJoiningDelay, setIsJoiningDelay] = useState(false)

We have our socket.on('connect' ...) listener on the client where we send the 'join_room' event to the server to join the client to a room. We fire off a useTimeout to do our minimum 800 millisecond delay that toggles isJoining to false.

As of socket.io version 4.4.0 there is now built in support for acknowledgement timeouts. We pass a timeout argument of 30000 milliseconds (30 seconds) which will throw an error if our server does not send back an acknowledgement within that time period. If the timeout it hit, the leaveRoom() function will disconnect the socket and redirect the user back to the login page where they can try to join again.

socket.on('connect', () => {
  setIsJoiningDelay(true)
  const joinRoom: JoinRoom = {
    roomName,
    user: { socketId: socket.id, ...user },
    eventName: 'join_room',
  }
  JoinRoomSchema.parse(joinRoom)
  setTimeout(() => {
    // default required 800 ms minimum join delay to prevent flickering
    setIsJoiningDelay(false)
  }, 800)
  socket.timeout(30000).emit('join_room', joinRoom, (err, response) => {
    if (err) {
      leaveRoom()
    }
    if (response) {
      setIsJoinedRoom(true)
    }
  })
  setIsConnected(true)
})

In the return of our chat page where we render UI elements, we have a top-level check if isJoinedRoom and !isJoiningDelay in which case we render the chat room components, and if not we reader a loading screen on the page.

return (
  <>
    {user?.userId && roomName && room && isJoinedRoom && !isJoiningDelay ? (
      <ChatLayout>
        <Header
          isConnected={isConnected}
          users={room?.users ?? []}
          roomName={roomName}
          handleUsersClick={() => setToggleUserList((toggleUserList) => !toggleUserList)}
          handleLeaveRoom={() => leaveRoom()}
        ></Header>
        {toggleUserList ? (
          <UserList
            room={room}
            currentUser={{ socketId: socket.id, ...user }}
            kickHandler={kickUser}
          ></UserList>
        ) : (
          <Messages user={user} messages={messages}></Messages>
        )}
        <MessageForm sendMessage={sendMessage}></MessageForm>
      </ChatLayout>
    ) : (
      <LoadingLayout>
        <Loading message={`Joining ${roomName}`}></Loading>
      </LoadingLayout>
    )}
  </>
)

Here is a snapshot of the loading screen.

So what about for our chat messages? How can we indicate to the user whether their messages are successfully being sent?

What we can do for messages is a delivered indicator alongside the messages in the UI.

To handle this, in our chat page we have a sendMessage() function that fires whenever a user sends a message.

Check if chat message passes schema check to ensure data is not malformed
Update state with the chat message but set it’s “delivered” status to false.
Emit the event and pass the acknowledgement callback function
When callback function is executed by the server, update the state of the previously pushed messages by setting the message’s “delivered” status to true.

const sendMessage = (message: string) => {
  if (user && socket && roomName) {
    const chatMessage: Message = {
      user: {
        userId: user.userId,
        userName: user.userName,
        socketId: socket.id,
      },
      timeSent: Date.now(),
      message,
      roomName: roomName,
      eventName: 'chat',
    }
    // Check if chat message passes schema check
    ChatMessageSchema.parse(chatMessage)
    // Update state with message "delivered" status to false
    setMessages((messages) => [{ ...chatMessage, delivered: false }, ...messages])
    // Emit 'chat' event with message and callback
    socket.emit('chat', chatMessage, (response) => {
      // If server response with response === true
      if (response) {
        // Update state by finding previously set message
        // and setting it's "delivered" status to true
        setMessages((messages) => {
          const previousMessageIndex = messages.findIndex((mes) => {
            if (mes.user.userId === user.userId && mes.timeSent === chatMessage.timeSent) {
              return mes
            }
          })
          if (previousMessageIndex === -1) {
            throw 'Previously sent message not found to update delivered status'
          }
          messages[previousMessageIndex] = {
            ...messages[previousMessageIndex],
            delivered: true,
          }
          return [...messages]
        })
      }
    })
  }
}

Now we can update our Messages component to display the “delivered” status of the messages.

We only want to display the status of the client’s own messages so we check if the client’s userId matches the message’s userId.

src/client/components/messages.tsx

import React from 'react'
import { Message, User } from '../../shared/interfaces/chat.interface'
export type ClientMessage = Message & { delivered: boolean }

const determineMessageStyle = (
  user: Pick<User, 'userId' | 'userName'>,
  messageUserId: Message['user']['userId']
) => {
  if (user && messageUserId === user.userId) {
    return {
      message: 'bg-slate-500 p-4 ml-24 rounded break-words',
      sender: 'ml-24 pl-4',
    }
  } else {
    return {
      message: 'bg-slate-800 p-4 mr-24 rounded break-words',
      sender: 'mr-24 pl-4',
    }
  }
}

export const Messages = ({
  user,
  messages,
}: {
  user: Pick<User, 'userId' | 'userName'>
  messages: ClientMessage[]
}) => {
  return (
    <div className="flex h-4/6 w-full flex-col-reverse overflow-y-scroll">
      {messages?.map((message, index) => {
        return (
          <div key={index + message.timeSent} className="mb-4">
            <div className={determineMessageStyle(user, message.user.userId).sender}>
              <span className="text-sm text-gray-400">{message.user.userName}</span>
              <span className="text-sm text-gray-400">{' ' + '•' + ' '}</span>
              <span className="text-sm text-gray-400">
                {new Date(message.timeSent).toLocaleString()}
              </span>
            </div>
            <div className={determineMessageStyle(user, message.user.userId).message}>
              <p className="text-white">{message.message}</p>
            </div>
            {user && message.user.userId === user.userId && (
              <p className="text-right text-xs text-gray-400">
                {message.delivered ? 'Delivered' : 'Not delivered'}
              </p>
            )}
          </div>
        )
      })}
    </div>
  )
}

Now if the user were to hit a rate limit on sending messages, they would know if the message was Not delivered or Delivered. Awesome!

Great! We added rate limiting to our websocket gateway, and introduced acknowledgements to send data back to the client to indicate what happened with their events. We could go much further with this but we’ll end it here for now!

Nest JS Websockets - Guards and Authorization with CASL

Delightful Engineering — Wed, 21 Dec 2022 17:20:27 +0000

Welcome to Part 4: Guards and Authorization with CASL, of building a realtime chat application with Nest JS and React.

In this part of the series, we’re going to introduce a new feature for the application which allows room hosts to kick users out of the room. For this, we want to implement a strategy for authorization to enable this feature on the server as we only want room hosts to be able to kick others - imagine a chat room where any user can kick any other user! That would be absolute chaos. We’ve also done some light refactoring on the server, added a new kick button and a new socket event to handle kicking users.

If you haven’t checkout out previous parts of this series, go back to follow along.

Part 1: Basics - laying the foundations of the application and implementing the most basic websockets features.
Part 2: Rooms - implementing websocket rooms and major UI improvements.
Part 3: Pipes and E2E Validation with Zod - validate mutations with Zod schemas shared between client and server. From form actions, to emitting client events, to handling events on the server - giving our application multiple levels of protection from malformed data.

This series will attempt to touch on as many features of Nest's websockets integration as well as socket.io in general.

Download the repository at this commit to follow along - we’re not going to cover everything to keep the focus on authorization.

Authorization

Authorization - not to be confused with authentication, is a process that determines what a user is able to do in an application. In this series part we are not covering authentication which more-so involves user identity, and general access to the application.

The nest documentation on authorization covers 3 strategies:

Basic RBAC implementation (role-based access control).
Claims-based authorization.
Attribute-based access control with the CASL library.

For our use case, allowing only users that are hosts of rooms to kick other users out of the room, the RBAC and claims-based authorization strategies aren’t going to be enough.

Attribute-based access control is a much more powerful strategy for building more context specific authorization cases so that’s what we’ll use.

We will be using the "@casl/ability": "^6.3.3" package to build out our new feature.

Guards

Guards in Nest JS are kind of like bouncers at a bar.

Before anyone goes into the bar, a bouncer might:

Check the person’s identification to make sure they’re old enough to drink alcohol (in the United States at least).
Check that the person is up to par with the dress code.
Make sure the person isn’t already too intoxicated.
Check any other arbitrary requirements they may have for entrance.

Guards are similar except for request data. Based on the rules of the authentication strategy, guards will typically get a reference to the client/user and the target resource, and run that relationship through the relevant rule checks. If one of those rule checks fail, the guard should fail with a forbidden exception - indicating that the client is forbidden from access or performing a certain action in relation to the resource.

Request Lifecycle

In the last part of this series we created pipes for our chat websocket gateway. Since we’ll now be adding a guard - we should review the request lifecycle for a Nest JS server.

The request lifecycle is has a bit more nuance than our graphic below, but for brevity this pretty much covers the flow. For more in depth on the request lifecycle check out the nest docs.

The pipes we added in the last part execute just before our gateway’s handler functions to validate the form of the input data.

We have yet to add any middleware or interceptors, but what we want now is a guard to validate that a user can access/mutate resources.

The Implementation

We will not be covering the entirety of the application to avoid rehashing all of the work done in the previous parts of the series - so let’s focus on what’s new.

Entities

Our chat application really has two entities - user and room. Even though we don’t have any kind of real database yet, we do have temporary data stores in the services themselves. If we were modeling a relational database for this application, we would most likely have at least two tables - a rooms table and a users table. Our entities would then describe the attributes that a row in each of the tables would consist of.

For this reason it makes sense for us to introduce these two entities.

First we have the roomentity which implements RoomType which we had already written before.

src/server/entities/room.entity.ts

import { RoomName, Room as RoomType, User } from '../../shared/interfaces/chat.interface'

export class Room implements RoomType {
  constructor(attrs: RoomType) {
    Object.assign(this, attrs)
  }
  name: RoomName
  host: User
  users: User[]
}

Secondly we have the user entity which implements UserType.

src/server/entities/user.entity.ts

import {
  SocketId,
  User as UserType,
  UserId,
  UserName,
} from '../../shared/interfaces/chat.interface'

export class User implements UserType {
  constructor(attrs: UserType) {
    Object.assign(this, attrs)
  }
  userId: UserId
  userName: UserName
  socketId: SocketId
}

Casl Ability Factory

With our services refactoring out of the way, let’s get to the main topic of this part of the series - the implementation of CASL to define some authorization rules.

As mentioned earlier, CASL employs an authorization strategy called attribute-based access control. This means we are able to build rules around specific attributes of entities.

src/server/casl/casl-ability.factory.ts

import {
  AbilityBuilder,
  createMongoAbility,
  ExtractSubjectType,
  InferSubjects,
  MongoAbility,
} from '@casl/ability'
import { Injectable } from '@nestjs/common'
import { Room } from '../entities/room.entity'
import { User } from '../entities/user.entity'

export enum Action {
  Kick = 'kick',
  Join = 'join',
  Message = 'message',
}

type Subjects = InferSubjects<typeof Room | typeof User> | 'all'
export type AppAbility = MongoAbility<[Action, Subjects]>
type FlatRoom = Room & {
  'host.userId': Room['host']['userId']
}

@Injectable()
export class CaslAbilityFactory {
  createForUser(user: User) {
    const { can, build } = new AbilityBuilder<AppAbility>(createMongoAbility)

    // Host can kick users from room
    can<FlatRoom>(Action.Kick, Room, {
      'host.userId': user.userId,
    })

    // Any user can join any room
    can(Action.Join, Room)

    // User can send messages in room given they are in the roo
    can(Action.Message, Room, {
      users: { $elemMatch: { userId: user.userId } },
    })

    return build({
      detectSubjectType: (object) => object.constructor as ExtractSubjectType<Subjects>,
    })
  }
}

Let’s step through our ability factory.

With CASL we should define a set of actions that can be performed on a set of subjects.

First we start out by defining an enum of the possible actions a user can perform.

A user can join, kick user from, and send message to a room.

Next we define some key types to use in the factory.

export enum Action {
  Kick = 'kick',
  Join = 'join',
  Message = 'message',
}

type Subjects = InferSubjects<typeof Room | typeof User> | 'all'
export type AppAbility = MongoAbility<[Action, Subjects]>
type FlatRoom = Room & {
  'host.userId': Room['host']['userId']
}

Our factory implements a createForUser method that takes a user. We’ll be able to use this method for users to define the abilities that they can have given certain attributes. CASL has a handy AbilityBuilder class that we can use that will provide us with can, cannot (although we aren’t using it), and build. With these we can define what a user can and cannot do.

Under the hood CASL uses MongoDB query language, although you don’t need to understand MongoDB and we of course aren’t using any databases - this is just a powerful way to create rules with entity attributes.

@Injectable()
export class CaslAbilityFactory {
  createForUser(user: User) {
    const { can, build } = new AbilityBuilder<AppAbility>(createMongoAbility)

    // Host can kick users from room
    can<FlatRoom>(Action.Kick, Room, {
      'host.userId': user.userId,
    })

    // Any user can join any room
    can(Action.Join, Room)

    // User can send messages in room given they are in the room
    can(Action.Message, Room, {
      users: { $elemMatch: { userId: user.userId } },
    })

    return build({
      detectSubjectType: (object) => object.constructor as ExtractSubjectType<Subjects>,
    })
  }
}

From our CASL module we need to make sure our factory is included in both the providers and the exports array. Including it in the exports array will allow us to inject it into other classes which we will do shortly.

src/server/casl/casl.module.ts

import { Module } from '@nestjs/common'
import { CaslAbilityFactory } from './casl-ability.factory'

@Module({
  providers: [CaslAbilityFactory],
  exports: [CaslAbilityFactory],
})
export class CaslModule {}

Finally we also have some policy types and interfaces that we will be using shortly.

src/server/casl/interfaces/policy.interface.ts

import { AppAbility } from '../casl-ability.factory'

interface IPolicyHandler {
  handle(ability: AppAbility): boolean
}

type PolicyHandlerCallback = (ability: AppAbility) => boolean

export type PolicyHandler = IPolicyHandler | PolicyHandlerCallback

Chat Policy Guard

Now that we have our ability factory, we can build a guard for out chat websocket gateway to process incoming payloads - and either reject the user from performing an action or let it pass through if the user is authorized.

src/server/chat/guards/chat.guard.ts

import { CanActivate, ExecutionContext, ForbiddenException, Injectable } from '@nestjs/common'
import { Action, AppAbility, CaslAbilityFactory } from '../../casl/casl-ability.factory'
import { RoomService } from '../../room/room.service'
import { PolicyHandler } from '../../casl/interfaces/policy.interface'
import {
  ClientToServerEvents,
  Room as RoomType,
  User,
} from '../../../shared/interfaces/chat.interface'
import { Room } from '../../entities/room.entity'

@Injectable()
export class ChatPoliciesGuard<
  CtxData extends {
    user: User
    roomName: RoomType['name']
    eventName: keyof ClientToServerEvents
  }
> implements CanActivate
{
  constructor(private caslAbilityFactory: CaslAbilityFactory, private roomService: RoomService) {}

  async canActivate(context: ExecutionContext): Promise<boolean> {
    const policyHandlers: PolicyHandler[] = []
    const ctx = context.switchToWs()
    const data = ctx.getData<CtxData>()
    const user = data.user
    const room = await this.roomService.getRoomByName(data.roomName)

    if (data.eventName === 'kick_user') {
      if (room === 'Not Exists') {
        throw `Room must exist to evaluate ${data.eventName} policy`
      }
      policyHandlers.push((ability) => ability.can(Action.Kick, room))
    }

    if (data.eventName === 'join_room') {
      policyHandlers.push((ability) => ability.can(Action.Join, Room))
    }

    if (data.eventName === 'chat') {
      if (room === 'Not Exists') {
        throw `Room must exist to evaluate ${data.eventName} policy`
      }
      policyHandlers.push((ability) => ability.can(Action.Message, room))
    }

    const ability = this.caslAbilityFactory.createForUser(user)
    policyHandlers.every((handler) => {
      const check = this.execPolicyHandler(handler, ability)
      if (check === false) {
        throw new ForbiddenException()
      }
    })
    return true
  }

  private execPolicyHandler(handler: PolicyHandler, ability: AppAbility) {
    if (typeof handler === 'function') {
      return handler(ability)
    }
    return handler.handle(ability)
  }
}

Our chat guard looks complex, but if we look at the code piece by piece it’s actually quite simple!

First, guards are injectable so we use the @Injectable() class decorator. We make this class a generic class by defining CtxData as a type parameter. We want to allow the instantiation to provide the type for context data in the instantiation of the class. With our caslAbilityFactory we’ll need the user(subject), the roomName (room is another subject) and the eventName (which will map to the actions enum we created earlier). To ensure the generic type parameter CtxData includes these fields, we apply the generic constraint with extends.

@Injectable()
export class ChatPoliciesGuard<
  CtxData extends {
    user: User;
    roomName: RoomType['name'];
    eventName: keyof ClientToServerEvents;
  },
> implements CanActivate
{
  constructor(
    private caslAbilityFactory: CaslAbilityFactory,
    private roomService: RoomService,
  ) {}
//.....

Guards implement a canActivate method that takes a context which in Nest terms in the execution context. This method will return a boolean - false indicates that the user did not pass the authorization checks, true indicates the user did pass the authorization checks. policyHandlers are an array of CASL ability function checks that we will use to do authorization rule checks. Since our application is a websockets application, we switch to a websockets context but setting const ctx = context.switchToWs();. This is important because now we can extract the incoming payload of the event which will give us access to all the data we need to check rules against the ability factory we created earlier. Based on the eventName we push an ability check handler to our policyHandlers, and at the end of the method we loop through the handlers. If one of the handlers returns false, we throw a new ForbiddenException.

async canActivate(context: ExecutionContext): Promise<boolean> {
    const policyHandlers: PolicyHandler[] = [];
    const ctx = context.switchToWs();
    const data = ctx.getData<CtxData>();
    const user = data.user;
    const room = await this.roomService.getRoomByName(data.roomName);

        if (data.eventName === 'kick_user') {
      if (room === 'Not Exists') {
        throw `Room must exist to evaluate ${data.eventName} policy`;
      }
      policyHandlers.push((ability) => ability.can(Action.Kick, room));
    }

    if (data.eventName === 'join_room') {
      policyHandlers.push((ability) => ability.can(Action.Join, Room));
    }

    if (data.eventName === 'chat') {
      if (room === 'Not Exists') {
        throw `Room must exist to evaluate ${data.eventName} policy`;
      }
      policyHandlers.push((ability) => ability.can(Action.Message, room));
    }

    const ability = this.caslAbilityFactory.createForUser(user);
    policyHandlers.every((handler) => {
      const check = this.execPolicyHandler(handler, ability);
      if (check === false) {
        throw new ForbiddenException();
      }
    });
    return true;
  }

Finally, here is the helper function we use to run the policy handler functions.

private execPolicyHandler(handler: PolicyHandler, ability: AppAbility) {
    if (typeof handler === 'function') {
      return handler(ability);
    }
    return handler.handle(ability);
  }

Chat Websocket Gateway

Again, it’s highly encouraged to go back to the last part of the series to get up to speed on how our chat gateway works - but here it is.

In our gateway event handler functions, we can now make use of the ChatPoliciesGuard that we created with @UseGuards(ChatPoliciesGuard<PayloadType>).

src/server/chat/chat.gateway.ts

import {
  MessageBody,
  SubscribeMessage,
  WebSocketGateway,
  WebSocketServer,
  OnGatewayConnection,
  OnGatewayDisconnect,
} from '@nestjs/websockets'
import { Logger, UseGuards, UsePipes } from '@nestjs/common'
import {
  ServerToClientEvents,
  ClientToServerEvents,
  Message,
  JoinRoom,
  KickUser,
} from '../../shared/interfaces/chat.interface'
import { Server, Socket } from 'socket.io'
import { RoomService } from '../room/room.service'
import { ZodValidationPipe } from '../pipes/zod.pipe'
import { ChatMessageSchema, JoinRoomSchema, KickUserSchema } from '../../shared/schemas/chat.schema'
import { UserService } from '../user/user.service'
import { ChatPoliciesGuard } from './guards/chat.guard'

@WebSocketGateway({
  cors: {
    origin: '*',
  },
})
export class ChatGateway implements OnGatewayConnection, OnGatewayDisconnect {
  constructor(private roomService: RoomService, private userService: UserService) {}

  @WebSocketServer() server: Server = new Server<ServerToClientEvents, ClientToServerEvents>()

  private logger = new Logger('ChatGateway')

  @UseGuards(ChatPoliciesGuard<Message>)
  @UsePipes(new ZodValidationPipe(ChatMessageSchema))
  @SubscribeMessage('chat')
  async handleChatEvent(
    @MessageBody()
    payload: Message
  ): Promise<void> {
    this.logger.log(payload)
    this.server.to(payload.roomName).emit('chat', payload) // broadcast messages
  }

  @UseGuards(ChatPoliciesGuard<JoinRoom>)
  @UsePipes(new ZodValidationPipe(JoinRoomSchema))
  @SubscribeMessage('join_room')
  async handleSetClientDataEvent(
    @MessageBody()
    payload: JoinRoom
  ): Promise<void> {
    if (payload.user.socketId) {
      this.logger.log(`${payload.user.socketId} is joining ${payload.roomName}`)
      await this.userService.addUser(payload.user)
      await this.server.in(payload.user.socketId).socketsJoin(payload.roomName)
      await this.roomService.addUserToRoom(payload.roomName, payload.user.userId)
    }
  }

  @UseGuards(ChatPoliciesGuard<KickUser>)
  @UsePipes(new ZodValidationPipe(KickUserSchema))
  @SubscribeMessage('kick_user')
  async handleKickUserEvent(@MessageBody() payload: KickUser): Promise<boolean> {
    this.logger.log(`${payload.userToKick.userName} is getting kicked from ${payload.roomName}`)
    await this.server.to(payload.roomName).emit('kick_user', payload)
    await this.server.in(payload.userToKick.socketId).socketsLeave(payload.roomName)
    await this.server.to(payload.roomName).emit('chat', {
      user: {
        userId: 'serverId',
        userName: 'TheServer',
        socketId: 'ServerSocketId',
      },
      timeSent: new Date(Date.now()).toLocaleString('en-US'),
      message: `${payload.userToKick.userName} was kicked.`,
      roomName: payload.roomName,
    })
    return true
  }

  async handleConnection(socket: Socket): Promise<void> {
    this.logger.log(`Socket connected: ${socket.id}`)
  }

  async handleDisconnect(socket: Socket): Promise<void> {
    const user = await this.roomService.getFirstInstanceOfUser(socket.id)
    if (user) {
      await this.userService.removeUserById(user.userId)
    }
    await this.roomService.removeUserFromAllRooms(socket.id)
    this.logger.log(`Socket disconnected: ${socket.id}`)
  }
}

Schemas and Interfaces

Now that our websocket message payloads must include the user, roomName, and eventName for our chat policies guard to work, we need to expand on our schemas and interfaces.

src/shared/schemas/chat.schema.ts

import { z } from 'zod'

export const UserIdSchema = z.string().min(1).max(24)

export const UserNameSchema = z
  .string()
  .min(1, { message: 'Must be at least 1 character.' })
  .max(16, { message: 'Must be at most 16 characters.' })

export const MessageSchema = z
  .string()
  .min(1, { message: 'Must be at least 1 character.' })
  .max(1000, { message: 'Must be at most 1000 characters.' })

export const TimeSentSchema = z.string()

export const RoomNameSchemaRegex = new RegExp('^\\S+\\w$')

export const RoomNameSchema = z
  .string()
  .min(2, { message: 'Must be at least 2 characters.' })
  .max(16, { message: 'Must be at most 16 characters.' })
  .regex(RoomNameSchemaRegex, {
    message: 'Must not contain spaces or special characters.',
  })

export const EventNameSchema = z.enum(['chat', 'kick_user', 'join_room'])

export const SocketIdSchema = z.string().length(20, { message: 'Must be 20 characters.' })

export const UserSchema = z.object({
  userId: UserIdSchema,
  userName: UserNameSchema,
  socketId: SocketIdSchema,
})

export const ChatMessageSchema = z.object({
  user: UserSchema,
  timeSent: TimeSentSchema,
  message: MessageSchema,
  roomName: RoomNameSchema,
  eventName: EventNameSchema,
})

export const RoomSchema = z.object({
  name: RoomNameSchema,
  host: UserSchema,
  users: UserSchema.array(),
})

export const JoinRoomSchema = z.object({
  user: UserSchema,
  roomName: RoomNameSchema,
  eventName: EventNameSchema,
})

export const KickUserSchema = z.object({
  user: UserSchema,
  userToKick: UserSchema,
  roomName: RoomNameSchema,
  eventName: EventNameSchema,
})

export const ClientToServerEventsSchema = z.object({
  chat: z.function().args(ChatMessageSchema).returns(z.void()),
  join_room: z.function().args(JoinRoomSchema).returns(z.void()),
  kick_user: z
    .function()
    .args(KickUserSchema, z.function().args(z.boolean()).returns(z.void()))
    .returns(z.void()),
})

export const ServerToClientEventsSchema = z.object({
  chat: z.function().args(ChatMessageSchema).returns(z.void()),
  kick_user: z.function().args(KickUserSchema).returns(z.void()),
})

src/shared/interfaces/chat.interface.ts

import { z } from 'zod'
import {
  ChatMessageSchema,
  JoinRoomSchema,
  KickUserSchema,
  RoomNameSchema,
  RoomSchema,
  SocketIdSchema,
  UserIdSchema,
  UserNameSchema,
  UserSchema,
  ServerToClientEventsSchema,
  ClientToServerEventsSchema,
} from '../schemas/chat.schema'

export type UserId = z.infer<typeof UserIdSchema>
export type UserName = z.infer<typeof UserNameSchema>
export type SocketId = z.infer<typeof SocketIdSchema>
export type User = z.infer<typeof UserSchema>

export type RoomName = z.infer<typeof RoomNameSchema>
export type Room = z.infer<typeof RoomSchema>
export type Message = z.infer<typeof ChatMessageSchema>

export type JoinRoom = z.infer<typeof JoinRoomSchema>
export type KickUser = z.infer<typeof KickUserSchema>

export type ServerToClientEvents = z.infer<typeof ServerToClientEventsSchema>
export type ClientToServerEvents = z.infer<typeof ClientToServerEventsSchema>

Frontend Client Modifications

The client now also needs to include user, roomName, and eventName when emitting events.

We’ll just show snippets of the frontend for brevity.

src/client/pages/chat.tsx

// 'join_room' event on socket connection
socket.on('connect', () => {
  const joinRoom: JoinRoom = {
    roomName,
    user: { socketId: socket.id, ...user },
    eventName: 'join_room',
  }
  JoinRoomSchema.parse(joinRoom)
  socket.emit('join_room', joinRoom)
  setIsConnected(true)
})
// .....
// 'chat' event when client sends a chat message
const sendMessage = (message: string) => {
  if (user && socket && roomName) {
    const chatMessage: Message = {
      user: {
        userId: user.userId,
        userName: user.userName,
        socketId: socket.id,
      },
      timeSent: new Date(Date.now()).toLocaleString('en-US'),
      message,
      roomName: roomName,
      eventName: 'chat',
    }
    ChatMessageSchema.parse(chatMessage)
    socket.emit('chat', chatMessage)
  }
}
//.....
// 'kick_user' event when host kicks a user
const kickUser = (userToKick: User) => {
  if (!room) {
    throw 'No room'
  }
  if (!user) {
    throw 'No current user'
  }
  const kickUserData: KickUser = {
    user: { ...user, socketId: socket.id },
    userToKick: userToKick,
    roomName: room.name,
    eventName: 'kick_user',
  }
  KickUserSchema.parse(kickUserData)
  socket.emit('kick_user', kickUserData, (complete) => {
    if (complete) {
      roomRefetch()
    }
  })
}

Also, in our <UserList/> component in our client where we display current users in the chat room, we’ve added a “Kick” button that will allow room hosts to kick users out of the room.

src/client/components/list.tsx

import React from 'react'
import { Room, User } from '../../shared/interfaces/chat.interface'

export const UserList = ({
  room,
  currentUser,
  kickHandler,
}: {
  room: Room
  currentUser: User
  kickHandler: (user: User) => void
}) => {
  return (
    <div className="flex h-4/6 w-full flex-col-reverse overflow-y-scroll">
      {room.users.map((user, index) => {
        return (
          <div key={index} className="mb-4 flex justify-between rounded px-4 py-2">
            <div className="flex items-center">
              <p className="text-white">{user.userName}</p>
              {room.host.userId === user.userId && <span className="ml-2">{'👑'}</span>}
            </div>
            {room.host.userId === currentUser.userId && user.userId !== currentUser.userId && (
              <button
                className="flex h-8 items-center justify-self-end rounded-xl bg-gray-800 px-4"
                onClick={() => kickHandler(user)}
              >
                <span className="mr-1 text-white">{'Kick'}</span>
              </button>
            )}
          </div>
        )
      })}
    </div>
  )
}

Here’s what that looks like now!

So in the above case, this is showing the host (Austin) client. Meeko’s client won’t have a kick button. But what if this application was growing, we hired a bunch of frontend developers, and they accidentally introduced a bug where Meeko now had a kick button.

Let’s simulate that and see what happens.

Oops! Meeko can now kick the host. That’s bad. But if he tries to…

[1] [Nest] 87703  - 12/21/2022, 11:28:09 AM   ERROR [WsExceptionsHandler] Forbidden
[1] ForbiddenException: Forbidden
[1]     at /Users/austinhoward/code/nest-realtime/nest-react-websockets/src/server/chat/guards/chat.guard.ts:64:15
[1]     at Array.every (<anonymous>)
[1]     at ChatPoliciesGuard.<anonymous> (/Users/austinhoward/code/nest-realtime/nest-react-websockets/src/server/chat/guards/chat.guard.ts:61:20)
[1]     at Generator.next (<anonymous>)
[1]     at fulfilled (/Users/austinhoward/code/nest-realtime/nest-react-websockets/dist/server/server/chat/guards/chat.guard.js:14:58)
[1]     at processTicksAndRejections (node:internal/process/task_queues:96:5)

Perfect! Our server is protected even if there is a mistake on the client.

That’s all for authorization. CASL provides us with powerful tools to composes situational authorization rules in our Nest application. With guards we were able to abstract these authorization capabilities out of our gateway in a clean and concise way.

Nest JS Websockets - Pipes and E2E Validation with Zod

Delightful Engineering — Sat, 10 Dec 2022 23:59:25 +0000

Welcome to Part 3: Pipes and E2E Validation with Zod, of building a realtime chat application with Nest JS and React.

If you haven’t checkout out previous parts of this series, go back to follow along.

Part 1: Basics - laying the foundations of the application and implementing the most basic websockets features.
Part 2: Rooms - implementing websocket rooms and major UI improvements.

This series will attempt to touch on as many features of Nest's websockets integration as well as socket.io in general.

Download the repository at this commit to follow along.

From part 2 of the series we’ve got the following features in our application now:

Users can create, join, and leave rooms.
Users can send messages in rooms.
Users can view other users currently in the room.

End to End Validation

While we have good types that our client and server share, we can do even better.

Not only do we want good types, we also want the ability to validate that our data is correct during runtime for both the frontend and backend.

Here’s what we can do:

Define schemas for our data with zod and use schema methods such as parse to do validation checks.
Use those schemas to infer types rather than explicitly writing them.
Create a custom pipe to do validation on the server with our schemas.
Use react-hook-form to enhance our login form and give us the ability to use our schemas to validate on form submissions.

Shared Schemas & Interfaces

Let’s start by looking at some updates to our src/shared directory.

We’ve added a new schemas directory and a new chat.schema.ts file. Now the structure looks like this:

interfaces
- chat.interface.ts
schemas
- chat.schema.ts

With zod, we’re able to create a schema for each field in our object schemas. We can put constraints around these fields and customize a message if these constraints fail when parsed at runtime.

shared/schemas/chat.schema.ts

import { z } from 'zod'

export const UserIdSchema = z.string().min(1).max(24)

export const UserNameSchema = z
  .string()
  .min(1, { message: 'Must be at least 1 character.' })
  .max(16, { message: 'Must be at most 16 characters.' })

export const MessageSchema = z
  .string()
  .min(1, { message: 'Must be at least 1 character.' })
  .max(1000, { message: 'Must be at most 1000 characters.' })

export const TimeSentSchema = z.string()

export const RoomNameSchemaRegex = new RegExp('^\\S+\\w$')

export const RoomNameSchema = z
  .string()
  .min(2, { message: 'Must be at least 2 characters.' })
  .max(16, { message: 'Must be at most 16 characters.' })
  .regex(RoomNameSchemaRegex, {
    message: 'Must not contain spaces or special characters.',
  })

export const SocketIdSchema = z.string().length(20, { message: 'Must be 20 characters.' })

export const UserSchema = z.object({
  userId: UserIdSchema,
  userName: UserNameSchema,
  socketId: SocketIdSchema,
})

export const ChatMessageSchema = z.object({
  user: UserSchema,
  timeSent: TimeSentSchema,
  message: MessageSchema,
  roomName: RoomNameSchema,
})

export const RoomSchema = z.object({
  name: RoomNameSchema,
  host: UserSchema,
  users: UserSchema.array(),
})

export const JoinRoomSchema = z.object({
  user: UserSchema,
  roomName: RoomNameSchema,
})

Now we’ve made some changes to our shared interfaces given that we now have schema for our data. We can use z.infer to infer the types from our schemas. This is AWESOME because now we can write our schemas once, infer types from them, and use them in both the frontend and backend.

shared/interfaces/chat.interface.ts

import { z } from 'zod'
import { ChatMessageSchema, JoinRoomSchema, RoomSchema, UserSchema } from '../schemas/chat.schema'

export type User = z.infer<typeof UserSchema>

export type Room = z.infer<typeof RoomSchema>

export type Message = z.infer<typeof ChatMessageSchema>

export type JoinRoom = z.infer<typeof JoinRoomSchema>

export interface ServerToClientEvents {
  chat: (e: Message) => void
}

export interface ClientToServerEvents {
  chat: (e: Message) => void
  join_room: (e: JoinRoom) => void
}

Server Validation

Let's start by looking at the backend first - our server layer.

Zod Object Schema Validation Pipe

Nest JS has a powerful and simple way to evaluate input data called pipes.

Here is the pipe that we’ve created. Our ZodValidationPipe accepts a constructor arg which is a zod schema. We then parse the schema which will either throw and error resulting in a WsException (websocket exception) if the data does not pass the schema constraints, otherwise the value passes through successfully.

server/pipes/zod.pipe.ts

import { PipeTransform, Injectable } from '@nestjs/common'
import { Schema } from 'zod'

@Injectable()
export class ZodValidationPipe implements PipeTransform {
  constructor(private schema: Schema) {}

  transform(value: any) {
    this.schema.parse(value)
    return value
  }
}

With our validation pipe and our schemas, we can now apply this to our websocket gateway.

server/chat/chat.gateway.ts

import {
  MessageBody,
  SubscribeMessage,
  WebSocketGateway,
  WebSocketServer,
  OnGatewayConnection,
  OnGatewayDisconnect,
} from '@nestjs/websockets'
import { Logger, UsePipes } from '@nestjs/common'
import {
  ServerToClientEvents,
  ClientToServerEvents,
  Message,
  JoinRoom,
} from '../../shared/interfaces/chat.interface'
import { Server, Socket } from 'socket.io'
import { UserService } from '../user/user.service'
import { ZodValidationPipe } from '../pipes/zod.pipe'
import { ChatMessageSchema, JoinRoomSchema } from '../../shared/schemas/chat.schema'

@WebSocketGateway({
  cors: {
    origin: '*',
  },
})
export class ChatGateway implements OnGatewayConnection, OnGatewayDisconnect {
  constructor(private userService: UserService) {}

  @WebSocketServer() server: Server = new Server<ServerToClientEvents, ClientToServerEvents>()

  private logger = new Logger('ChatGateway')

  @UsePipes(new ZodValidationPipe(ChatMessageSchema))
  @SubscribeMessage('chat')
  async handleChatEvent(
    @MessageBody()
    payload: Message
  ): Promise<void> {
    this.logger.log(payload)
    this.server.to(payload.roomName).emit('chat', payload) // broadcast messages
  }

  @UsePipes(new ZodValidationPipe(JoinRoomSchema))
  @SubscribeMessage('join_room')
  async handleSetClientDataEvent(
    @MessageBody()
    payload: JoinRoom
  ): Promise<void> {
    if (payload.user.socketId) {
      this.logger.log(`${payload.user.socketId} is joining ${payload.roomName}`)
      await this.server.in(payload.user.socketId).socketsJoin(payload.roomName)
      await this.userService.addUserToRoom(payload.roomName, payload.user)
    }
  }

  async handleConnection(socket: Socket): Promise<void> {
    this.logger.log(`Socket connected: ${socket.id}`)
  }

  async handleDisconnect(socket: Socket): Promise<void> {
    await this.userService.removeUserFromAllRooms(socket.id)
    this.logger.log(`Socket disconnected: ${socket.id}`)
  }
}

Let’s drill in.

We’re able to use the built in @UsePipes decorator to instantiate our new pipe and pass in the ChatMessageSchema.

If the incoming message from a client does not pass the ChatMessageSchema constraints, and error will be throw and the message will not get emitted. This will protect our server from processing bad messages.

@UsePipes(new ZodValidationPipe(ChatMessageSchema))
@SubscribeMessage('chat')
async handleChatEvent(
  @MessageBody()
  payload: Message,
): Promise<void> {
  this.logger.log(payload);
  this.server.to(payload.roomName).emit('chat', payload); // broadcast messages
}

One additional nicety here is that our Message type we’re giving the payload is also derived from our ChatMessageSchema which we saw earlier but we’ll show again below from our shared interfaces.

export type Message = z.infer<typeof ChatMessageSchema>

If we wanted to add new fields to our chat messages, all we would need to do is update the ChatMessageSchema.

For users joining rooms, we’re also validating that data as well.

@UsePipes(new ZodValidationPipe(JoinRoomSchema))
@SubscribeMessage('join_room')
async handleSetClientDataEvent(
  @MessageBody()
  payload: JoinRoom,
): Promise<void> {
  if (payload.user.socketId) {
    this.logger.log(
      `${payload.user.socketId} is joining ${payload.roomName}`,
    );
    await this.server.in(payload.user.socketId).socketsJoin(payload.roomName);
    await this.userService.addUserToRoom(payload.roomName, payload.user);
  }
}

Here’s an example of a pipe throwing an exception. We had to modify the schema to show this, but this would happen given that the chat message is sent to the server in a malformed way:

[1] [Nest] 18224  - 12/10/2022, 1:51:47 PM   ERROR [WsExceptionsHandler] [
[1]   {
[1]     "code": "too_small",
[1]     "minimum": 2,
[1]     "type": "string",
[1]     "inclusive": true,
[1]     "message": "Must be at least 1 character.",
[1]     "path": [
[1]       "message"
[1]     ]
[1]   }
[1] ]
[1] ZodError: [
[1]   {
[1]     "code": "too_small",
[1]     "minimum": 2,
[1]     "type": "string",
[1]     "inclusive": true,
[1]     "message": "Must be at least 1 character.",
[1]     "path": [
[1]       "message"
[1]     ]
[1]   }
[1] ]
[1]     at handleResult (/Users/austinhoward/code/nest-realtime/nest-react-websockets/node_modules/zod/lib/types.js:29:23)
[1]     at ZodObject.safeParse (/Users/austinhoward/code/nest-realtime/nest-react-websockets/node_modules/zod/lib/types.js:140:16)
[1]     at ZodObject.parse (/Users/austinhoward/code/nest-realtime/nest-react-websockets/node_modules/zod/lib/types.js:120:29)
[1]     at ZodValidationPipe.transform (/Users/austinhoward/code/nest-realtime/nest-react-websockets/src/server/pipes/zod.pipe.ts:9:17)
[1]     at /Users/austinhoward/code/nest-realtime/nest-react-websockets/node_modules/@nestjs/core/pipes/pipes-consumer.js:16:33
[1]     at processTicksAndRejections (node:internal/process/task_queues:96:5)

Client Validation

Previously we were not using any special libraries for the forms in our React client.

There are several good form libraries, notably React Hook Form and Formik. We’re going to use React Hook Form.

Let’s start with our login form. We’re using a mixture of React Hook Form validation and native HTML form validation techniques to give us a very robust form.

client/components/login.form.tsx

import React from 'react'
import { useForm } from 'react-hook-form'
import { zodResolver } from '@hookform/resolvers/zod'
import { z } from 'zod'
import {
  RoomNameSchema,
  UserNameSchema,
  RoomNameSchemaRegex,
} from '../../shared/schemas/chat.schema'
import { generateUserId, setUser } from '../lib/user'

const formSchema = z.object({
  userName: UserNameSchema,
  roomName: RoomNameSchema.or(z.string().length(0))
    .optional()
    .transform((name) => (name === '' ? undefined : name)),
})

export type LoginFormInputs = z.infer<typeof formSchema>

export const LoginForm = ({
  onSubmitSecondary,
  disableNewRoom,
  defaultUser,
}: {
  onSubmitSecondary: (data: LoginFormInputs) => void
  disableNewRoom: boolean
  defaultUser?: string
}) => {
  const {
    register,
    handleSubmit,
    formState: { errors },
  } = useForm<LoginFormInputs>({
    resolver: zodResolver(formSchema),
    mode: 'onChange',
  })

  const onSubmit = (data: LoginFormInputs) => {
    const newUser = {
      id: generateUserId(data.userName),
      name: data.userName,
    }
    setUser(newUser)
    onSubmitSecondary(data)
  }

  return (
    <div className="h-full w-full py-2 md:px-2 md:py-0">
      <form onSubmit={handleSubmit(onSubmit)} className="flex flex-col justify-center">
        <input
          type="text"
          id="login"
          placeholder="Name"
          defaultValue={defaultUser && defaultUser}
          required={true}
          minLength={UserNameSchema.minLength ?? undefined}
          maxLength={UserNameSchema.maxLength ?? undefined}
          {...register('userName')}
          className="h-12 rounded-md border border-slate-400 bg-gray-800 text-white placeholder-slate-400 invalid:text-pink-600 invalid:ring-pink-600 focus:border-violet-500 focus:outline-none focus:ring-1 focus:ring-violet-500 focus:invalid:border-pink-600 focus:invalid:ring-pink-600 active:invalid:border-pink-600"
        ></input>
        <p className="py-1 text-sm text-pink-600">{errors.userName?.message}</p>
        <input
          type="text"
          id="room"
          required={!disableNewRoom}
          disabled={disableNewRoom}
          minLength={RoomNameSchema?.minLength ?? undefined}
          maxLength={RoomNameSchema?.maxLength ?? undefined}
          pattern={RoomNameSchemaRegex.source.toString()}
          placeholder="New room"
          {...register('roomName')}
          className="h-12 rounded-md border border-slate-400 bg-gray-800 text-white placeholder-slate-400 invalid:text-pink-600 invalid:ring-pink-600 focus:border-violet-500 focus:outline-none focus:ring-1 focus:ring-violet-500 focus:invalid:border-pink-600 focus:invalid:ring-pink-600 disabled:opacity-50"
        ></input>
        <p className="py-1 text-sm text-pink-600">{errors.roomName?.message}</p>

        <button
          type="submit"
          className="flex h-12 w-full items-center justify-center rounded-md bg-violet-700 text-white"
        >
          Join
        </button>
      </form>
    </div>
  )
}

We’ll go step by step through the form updates.

First, we’re constructing a schema specific to the form’s input values - formSchema.

The two form input values:

userName: takes the UserNameSchema from our shared interfaces.
roomName: takes the RoomNameSchema from our shared interfaces. Since this field is specifically for creating a new room, there is a scenario where the user instead selects an existing room instead of creating a new one, in which case we need to account for the roomName being a string of length 0. We handle this by chaining .or(z.string().length(0), with .optional(), and finally with a transform that will make the field undefined if the string is of length 0 .transform((name) => (name === '' ? undefined : name).

const formSchema = z.object({
  userName: UserNameSchema,
  roomName: RoomNameSchema.or(z.string().length(0))
    .optional()
    .transform((name) => (name === '' ? undefined : name)),
})

We then create the form inputs type by inference.

export type LoginFormInputs = z.infer<typeof formSchema>

Coming inside the form we need to bring in useForm from React Hook Form and pass in our LoginFormInputs type. We’re going to use the following return values and options:

Returns

register: allows us to register form input elements for validation.
handleSubmit: allows us to capture form data if validation is successful.
formState: contains information about the state of the form, we can use this to grab errors.

Options

resolver: which is going to handle our form validation. Since we’re using zod, we use zodResolver and pass in our formSchema.
mode: this option allows us to configure when the form validation will happen - in this case we’ve chosen onChange so that every time a change event happens on the form, we’re executing a validation.

const {
  register,
  handleSubmit,
  formState: { errors },
} = useForm<LoginFormInputs>({
  resolver: zodResolver(formSchema),
  mode: 'onChange',
})

We need to have a function to execute on submission as well - where we can also pass in a data argument of LoginFormInputs type.

const onSubmit = (data: LoginFormInputs) => {
  const newUser = {
    id: generateUserId(data.userName),
    name: data.userName,
  }
  setUser(newUser)
  onSubmitSecondary(data)
}

Our name input form element also has some native form validation constraints which are required, minLength, and maxLength. We’re conveniently able to use our schema values for the length constraints which provides us extra validation protection.

We also render any errors below the input element.

<input
  type="text"
  id="login"
  placeholder="Name"
  defaultValue={defaultUser && defaultUser}
  required={true}
  minLength={UserNameSchema.minLength ?? undefined}
  maxLength={UserNameSchema.maxLength ?? undefined}
  {...register('userName')}
  className="h-12 rounded-md border border-slate-400 bg-gray-800 text-white placeholder-slate-400 invalid:text-pink-600 invalid:ring-pink-600 focus:border-violet-500 focus:outline-none focus:ring-1 focus:ring-violet-500 focus:invalid:border-pink-600 focus:invalid:ring-pink-600 active:invalid:border-pink-600"
></input>
<p className="py-1 text-sm text-pink-600">{errors.userName?.message}</p>

Our room input element also uses some native form validation constraints. The new interesting one here is pattern which we’re also able to use from our RoomNameSchemaRegex.

<input
  type="text"
  id="room"
  required={!disableNewRoom}
  disabled={disableNewRoom}
  minLength={RoomNameSchema?.minLength ?? undefined}
  maxLength={RoomNameSchema?.maxLength ?? undefined}
  pattern={RoomNameSchemaRegex.source.toString()}
  placeholder="New room"
  {...register('roomName')}
  className="h-12 rounded-md border border-slate-400 bg-gray-800 text-white placeholder-slate-400 invalid:text-pink-600 invalid:ring-pink-600 focus:border-violet-500 focus:outline-none focus:ring-1 focus:ring-violet-500 focus:invalid:border-pink-600 focus:invalid:ring-pink-600 disabled:opacity-50"
></input>
<p className="py-1 text-sm text-pink-600">{errors.roomName?.message}</p>

Here are some snapshots of validation errors in our login form.

Forms are our first layer of protection from malformed data on in the client, but we have added one more secondary layer as well in our Chat page.

client/pages/chat.tsx

import React, { useState, useEffect } from 'react'
import { MakeGenerics, useMatch, useNavigate } from '@tanstack/react-location'
import { io, Socket } from 'socket.io-client'
import {
  User,
  Message,
  ServerToClientEvents,
  ClientToServerEvents,
} from '../../shared/interfaces/chat.interface'
import { Header } from '../components/header'
import { UserList } from '../components/list'
import { MessageForm } from '../components/message.form'
import { Messages } from '../components/messages'
import { ChatLayout } from '../layouts/chat.layout'
import { unsetRoom, useRoomQuery } from '../lib/room'
import { getUser } from '../lib/user'
import { ChatMessageSchema, JoinRoomSchema } from '../../shared/schemas/chat.schema'

const socket: Socket<ServerToClientEvents, ClientToServerEvents> = io({
  autoConnect: false,
})

function Chat() {
  const {
    data: { user, roomName },
  } = useMatch<ChatLocationGenerics>()

  const [isConnected, setIsConnected] = useState(socket.connected)
  const [messages, setMessages] = useState<Message[]>([])
  const [toggleUserList, setToggleUserList] = useState<boolean>(false)

  const { data: room } = useRoomQuery(roomName, isConnected)

  const navigate = useNavigate()

  useEffect(() => {
    if (!user || !roomName) {
      navigate({ to: '/', replace: true })
    } else {
      socket.on('connect', () => {
        const joinRoom = {
          roomName,
          user: { socketId: socket.id, ...user },
        }
        JoinRoomSchema.parse(joinRoom)
        socket.emit('join_room', joinRoom)
        setIsConnected(true)
      })

      socket.on('disconnect', () => {
        setIsConnected(false)
      })

      socket.on('chat', (e) => {
        setMessages((messages) => [e, ...messages])
      })

      socket.connect()
    }
    return () => {
      socket.off('connect')
      socket.off('disconnect')
      socket.off('chat')
    }
  }, [])

  const leaveRoom = () => {
    socket.disconnect()
    unsetRoom()
    navigate({ to: '/', replace: true })
  }

  const sendMessage = (message: string) => {
    if (user && socket && roomName) {
      const chatMessage = {
        user: {
          userId: user.userId,
          userName: user.userName,
          socketId: socket.id,
        },
        timeSent: new Date(Date.now()).toLocaleString('en-US'),
        message,
        roomName: roomName,
      }
      ChatMessageSchema.parse(chatMessage)
      socket.emit('chat', chatMessage)
    }
  }
  return (
    <>
      {user?.userId && roomName && room && (
        <ChatLayout>
          <Header
            isConnected={isConnected}
            users={room?.users ?? []}
            roomName={roomName}
            handleUsersClick={() => setToggleUserList((toggleUserList) => !toggleUserList)}
            handleLeaveRoom={() => leaveRoom()}
          ></Header>
          {toggleUserList ? (
            <UserList room={room}></UserList>
          ) : (
            <Messages user={user} messages={messages}></Messages>
          )}
          <MessageForm sendMessage={sendMessage}></MessageForm>
        </ChatLayout>
      )}
    </>
  )
}

export const loader = async () => {
  const user = getUser()
  return {
    user: user,
    roomName: sessionStorage.getItem('room'),
  }
}

type ChatLocationGenerics = MakeGenerics<{
  LoaderData: {
    user: Pick<User, 'userId' | 'userName'>
    roomName: string
  }
}>

export default Chat

We went over the Chat page in depth in the previous series part so we’ll just focus on the schema validation additions.

Whenever a client socket connects, we emit a 'join_room' event. Now we’ve added a JoinRoomSchema.parse(joinRoom) which will throw an error if it fails the schema checks, otherwise the event will fire successfully.

socket.on('connect', () => {
  const joinRoom = {
    roomName,
    user: { socketId: socket.id, ...user },
  }
  JoinRoomSchema.parse(joinRoom)
  socket.emit('join_room', joinRoom)
  setIsConnected(true)
})

Similarly in our sendMessage function we’re using ChatMessageSchema.parse(chatMessage) to validate the message data as well.

const sendMessage = (message: string) => {
  if (user && socket && roomName) {
    const chatMessage = {
      user: {
        userId: user.userId,
        userName: user.userName,
        socketId: socket.id,
      },
      timeSent: new Date(Date.now()).toLocaleString('en-US'),
      message,
      roomName: roomName,
    }
    ChatMessageSchema.parse(chatMessage)
    socket.emit('chat', chatMessage)
  }
}

Here’s an example of a failure which is caught in the client before ever getting to the server. We had to modify the schema to simulate this error:

Conclusion

This wraps up how we can use pipes, zod, and React Hook Form to achieve end-to-end validation of data.

We now have multiple layers of protection and can have greater confidence that our application will operate as intended.

We were able to write our schemas in one place and use them everywhere throughout our applications!

Nest JS Websockets - Rooms

Delightful Engineering — Thu, 08 Dec 2022 16:44:10 +0000

Welcome to Part 2: Rooms, of building a realtime chat application with Nest JS and React.

If you haven't checked out Part 1: Basics, go back to follow along as we've laid the foundations of the application there. We'll pick up where we left off there.

This series will attempt to touch on as many features of Nest's websockets integration as well as socket.io in general.

Download the repository at this commit to follow along.

What we have now from Part 1 is a basic chat application where all connected clients are in the same global chat, in other words everyone connected to the server will receive everyone's messages.

So what we want to do now is take advantage of rooms, is a key feature of socket.io which is our websockets framework of choice.

Server Updates

We need to update our chat websocket gateway to subscribe to a new event that will allow users to join rooms.

Before we make any updates to our gateway, we need to create a new module to support user functionality.

To do this quickly, you can use the Nest CLI:

nest generate module user
nest generate service user
nest generate controller user

Now, the user module should look like this:

server/user/user.module.ts

import { Module } from '@nestjs/common'
import { UserController } from './user.controller'
import { UserService } from './user.service'

@Module({
  controllers: [UserController],
  providers: [UserService],
  exports: [UserService],
})
export class UserModule {}

The two new important things we need to support rooms is our user.service.ts and user.controller.ts files.

Shared Types

First let’s look at our shared types in src/shared.

shared/interfaces/chat.interface.ts

export interface User {
  userId: string
  userName: string
  socketId: string
}

export interface Room {
  name: string
  host: User
  users: User[]
}

export interface Message {
  user: User
  timeSent: string
  message: string
  roomName: string
}

export interface ServerToClientEvents {
  chat: (e: Message) => void
}

export interface ClientToServerEvents {
  chat: (e: Message) => void
  join_room: (e: { user: User; roomName: string }) => void
}

The new type updates are that we introduce a new socketId type to the User interface as we’ll need to associate a unique socket to users.

We also have a new Room type where rooms will have a name, a host which will just be the user who created the room, and a list of users in the room.

We’re adding one more required type to our Message interface which is roomName since now all messages sent will be sent to a specific room (one room at a time for now.)

Finally we’re adding one more required type to our ClientToServerEvents interface as we’ll need clients to emit events to the server when they’re joining rooms.

User Service

server/user/user.service.ts

import { Injectable } from '@nestjs/common'
import { Room, User } from '../../shared/interfaces/chat.interface'

@Injectable()
export class UserService {
  private rooms: Room[] = []

  async addRoom(roomName: string, host: User): Promise<void> {
    const room = await this.getRoomByName(roomName)
    if (room === -1) {
      await this.rooms.push({ name: roomName, host, users: [host] })
    }
  }

  async removeRoom(roomName: string): Promise<void> {
    const findRoom = await this.getRoomByName(roomName)
    if (findRoom !== -1) {
      this.rooms = this.rooms.filter((room) => room.name !== roomName)
    }
  }

  async getRoomHost(hostName: string): Promise<User> {
    const roomIndex = await this.getRoomByName(hostName)
    return this.rooms[roomIndex].host
  }

  async getRoomByName(roomName: string): Promise<number> {
    const roomIndex = this.rooms.findIndex((room) => room?.name === roomName)
    return roomIndex
  }

  async addUserToRoom(roomName: string, user: User): Promise<void> {
    const roomIndex = await this.getRoomByName(roomName)
    if (roomIndex !== -1) {
      this.rooms[roomIndex].users.push(user)
      const host = await this.getRoomHost(roomName)
      if (host.userId === user.userId) {
        this.rooms[roomIndex].host.socketId = user.socketId
      }
    } else {
      await this.addRoom(roomName, user)
    }
  }

  async findRoomsByUserSocketId(socketId: string): Promise<Room[]> {
    const filteredRooms = this.rooms.filter((room) => {
      const found = room.users.find((user) => user.socketId === socketId)
      if (found) {
        return found
      }
    })
    return filteredRooms
  }

  async removeUserFromAllRooms(socketId: string): Promise<void> {
    const rooms = await this.findRoomsByUserSocketId(socketId)
    for (const room of rooms) {
      await this.removeUserFromRoom(socketId, room.name)
    }
  }

  async removeUserFromRoom(socketId: string, roomName: string): Promise<void> {
    const room = await this.getRoomByName(roomName)
    this.rooms[room].users = this.rooms[room].users.filter((user) => user.socketId !== socketId)
    if (this.rooms[room].users.length === 0) {
      await this.removeRoom(roomName)
    }
  }

  async getRooms(): Promise<Room[]> {
    return this.rooms
  }
}

A lot of these class methods are pretty straightforward and are in one way or another doing sorts of getters, setters, or directly mutating the rooms array which stores all active rooms on the server.

User Controller

We need to create a controller for interacting with our user service via http as we’ll want to ping the server less often to retrieve all or specific rooms.

server/user/user.controller.ts

import { Controller, Get, Param } from '@nestjs/common'
import { Room } from '../../shared/interfaces/chat.interface'
import { UserService } from './user.service'

@Controller()
export class UserController {
  constructor(private userService: UserService) {}

  @Get('api/rooms')
  async getAllRooms(): Promise<Room[]> {
    return await this.userService.getRooms()
  }

  @Get('api/rooms/:room')
  async getRoom(@Param() params): Promise<Room> {
    const rooms = await this.userService.getRooms()
    const room = await this.userService.getRoomByName(params.room)
    return rooms[room]
  }
}

Our server will now also have two GET routes to fetch active room data from which the clients can fetch.

api/rooms: will retrieve all active rooms.
api/rooms/:room: will retrieve one room via a room parameter.

Here are some sample JSONs from these routes to get a picture of the data structure.

api/rooms response

[
  {
    "name": "secretroom",
    "host": {
      "socketId": "n6wiTBGp5Cja8F0qAAAB",
      "userId": "1,670,466,928,442Anna",
      "userName": "Anna"
    },
    "users": [
      {
        "socketId": "n6wiTBGp5Cja8F0qAAAB",
        "userId": "1,670,466,928,442Anna",
        "userName": "Anna"
      },
      {
        "socketId": "58r6srrtvBQP-o1pAAAD",
        "userId": "1,670,463,510,408Austin",
        "userName": "Austin"
      }
    ]
  },
  {
    "name": "meekosroom",
    "host": {
      "socketId": "5orXGez8W05N793CAAAF",
      "userId": "1,670,510,121,768Meeko",
      "userName": "Meeko"
    },
    "users": [
      {
        "socketId": "5orXGez8W05N793CAAAF",
        "userId": "1,670,510,121,768Meeko",
        "userName": "Meeko"
      }
    ]
  }
]

api/rooms/secretroom response

{
  "name": "secretroom",
  "host": {
    "socketId": "n6wiTBGp5Cja8F0qAAAB",
    "userId": "1,670,466,928,442Anna",
    "userName": "Anna"
  },
  "users": [
    {
      "socketId": "n6wiTBGp5Cja8F0qAAAB",
      "userId": "1,670,466,928,442Anna",
      "userName": "Anna"
    },
    {
      "socketId": "58r6srrtvBQP-o1pAAAD",
      "userId": "1,670,463,510,408Austin",
      "userName": "Austin"
    }
  ]
}

Chat Gateway

We’ll also be making a few updates to our existing chat gateway file so that we can support rooms.

server/chat/chat.gateway.ts

import {
  MessageBody,
  SubscribeMessage,
  WebSocketGateway,
  WebSocketServer,
  OnGatewayConnection,
  OnGatewayDisconnect,
} from '@nestjs/websockets'
import { Logger } from '@nestjs/common'
import {
  ServerToClientEvents,
  ClientToServerEvents,
  Message,
  User,
} from '../../shared/interfaces/chat.interface'
import { Server, Socket } from 'socket.io'
import { UserService } from '../user/user.service'

@WebSocketGateway({
  cors: {
    origin: '*',
  },
})
export class ChatGateway implements OnGatewayConnection, OnGatewayDisconnect {
  constructor(private userService: UserService) {}

  @WebSocketServer() server: Server = new Server<ServerToClientEvents, ClientToServerEvents>()

  private logger = new Logger('ChatGateway')

  @SubscribeMessage('chat')
  async handleChatEvent(
    @MessageBody()
    payload: Message
  ): Promise<Message> {
    this.logger.log(payload)
    this.server.to(payload.roomName).emit('chat', payload) // broadcast messages
    return payload
  }

  @SubscribeMessage('join_room')
  async handleSetClientDataEvent(
    @MessageBody()
    payload: {
      roomName: string
      user: User
    }
  ) {
    if (payload.user.socketId) {
      this.logger.log(`${payload.user.socketId} is joining ${payload.roomName}`)
      await this.server.in(payload.user.socketId).socketsJoin(payload.roomName)
      await this.userService.addUserToRoom(payload.roomName, payload.user)
    }
  }

  async handleConnection(socket: Socket): Promise<void> {
    this.logger.log(`Socket connected: ${socket.id}`)
  }

  async handleDisconnect(socket: Socket): Promise<void> {
    await this.userService.removeUserFromAllRooms(socket.id)
    this.logger.log(`Socket disconnected: ${socket.id}`)
  }
}

There are some important new additions to our gateway.

Our ChatGateway class now implements OnGatewayConnection, and OnGatewayDisconnect which are two handy lifecycle hooks provided by Nest out of the box. With this we have two new functions to handle when there are socket connections or disconnections from clients to the server.

// Will fire when a client connects to the server
async handleConnection(socket: Socket): Promise<void> {
    this.logger.log(`Socket connected: ${socket.id}`);
}

// Will fire when a client disconnects from the server
async handleDisconnect(socket: Socket): Promise<void> {
  await this.userService.removeUserFromAllRooms(socket.id);
  this.logger.log(`Socket disconnected: ${socket.id}`);
}

Subscribe to a 'join_room' message which clients will send when they’re joining rooms. The socket.io specific important function in here is the socketsJoin where will join a user’s socketId with the room they’re joining which is all sent through the payload.

await this.server.in(payload.user.socketId).socketsJoin(payload.roomName)

Then we need to add the user to the internal state of our user service.

await this.userService.addUserToRoom(payload.roomName, payload.user)

Note this is all being stored in memory on the server, if we were wanting to scale this up we may want to add an adapter for a dedicated data store like redis which we’ll do in later parts of this series.

Client Updates

A couple of important new concepts we’re adding to our client:

Client-side routing with React Location.
Client-server state management with React Query.
Joining / leaving rooms and viewing users in the room.

App Entry Point and Routes

The application entry point (index.tsx and app.tsx) is now responsible for only core high level.

Our index.tsx hasn’t changed.

client/index.tsx

import React from 'react'
import ReactDOM from 'react-dom'
import App from './app'
import './styles/main.css'

ReactDOM.render(<App />, document.getElementById('root'))

Our core app file previously contained the entirety of the user interface. Now since we’re introducing client-side routing and state management, we’ve trimmed down this file to contain a few things:

queryClient: used to interact with the React Query cache.
QueryClientProvider: provider to connect and provide the query client to the application which uses React context under the hood.
AppRouter: our application’s router which we’ve created to route between different pages.

client/app.tsx

import React from 'react'
import { QueryClient, QueryClientProvider } from '@tanstack/react-query'
import { AppRouter } from './routes'

const queryClient = new QueryClient()

function App() {
  return (
    <QueryClientProvider client={queryClient}>
      <AppRouter />
    </QueryClientProvider>
  )
}

export default App

We’ve created our own file dedicated to routing. To use React Location we have to instantiate the ReactLocation class and pass location into the Router component that we’re returning from our AppRouter.

One particularly powerful feature of React Location is Route Loaders. For both of our routes we’re passing:

path: url path to the route.
element: react component (page) to render.
loader: a function to fetch some data at the route level which is made available to the page we’ll see where we grab this loader data shortly when we get to our pages. This is great because now we can fetch data as early as possible.

The <Outlet/> component is where our routes will start rendering.

client/routes.tsx

import React from 'react'
import { Outlet, ReactLocation, Router } from '@tanstack/react-location'
import Login, { loader as loginLoader } from './pages/login'
import Chat, { loader as chatLoader } from './pages/chat'

const location = new ReactLocation()

export const AppRouter = () => {
  return (
    <Router
      location={location}
      routes={[
        {
          path: '/',
          element: <Login />,
          loader: async () => await loginLoader(),
        },
        {
          path: 'chat',
          element: <Chat />,
          loader: async () => await chatLoader(),
        },
      ]}
    >
      <Outlet />
    </Router>
  )
}

Layouts

Layouts are a great way to position content on pages at a high level.

Nothing particularly notable about these layouts other than that they take advantage of composition (containment) by passing children.

client/layouts/login.layout.tsx

import React from 'react'

export const LoginLayout = ({ children }: { children: React.ReactElement[] }) => {
  return (
    <div className="mx-auto flex h-screen w-screen justify-center bg-gray-900">
      <div className="h-127 w-127 my-auto flex flex-col p-2 md:flex-row">{children}</div>
    </div>
  )
}

Another thing to mention about layouts is that this is a good place to make your application responsive. You’ll see that with tailwind we’ve used size prefixes like md:, lg:, xl: to target different screen sizes.

client/layouts/chat.layout.tsx

import React from 'react'

export const ChatLayout = ({ children }: { children: React.ReactElement[] }) => {
  return (
    <div className="mx-auto flex h-screen w-screen justify-center bg-gray-900">
      <div className="flex h-full w-full flex-col px-2 md:w-8/12 lg:w-6/12 xl:w-4/12">
        {children}
      </div>
    </div>
  )
}

Pages

We’re splitting up our login and chat views into pages that can be routed to.

We’ll create a new /pages directory under /client and create two new pages.

client/pages/login.tsx

import React, { useEffect, useState } from 'react'
import { MakeGenerics, useMatch, useNavigate } from '@tanstack/react-location'
import { User } from '../../shared/interfaces/chat.interface'
import { LoginForm } from '../components/login.form'
import { Rooms } from '../components/rooms'
import { LoginLayout } from '../layouts/login.layout'
import { useRoomsQuery } from '../lib/room'
import { generateUserId, getUser, setUser } from '../lib/user'

function Login() {
  const {
    data: { user, roomName },
  } = useMatch<LoginLocationGenerics>()

  const [joinRoomSelection, setJoinRoomSelection] = useState<string>('')

  const { data: rooms, isLoading: roomsLoading } = useRoomsQuery()

  const navigate = useNavigate()

  const login = (e: React.FormEvent<HTMLFormElement>) => {
    const userFormValue = e.target[0].value
    const roomFormValue = e.target[1].value
    const newUser = {
      userId: generateUserId(userFormValue),
      userName: userFormValue,
    }
    setUser({ id: newUser.userId, name: newUser.userName })
    if (joinRoomSelection !== '') {
      sessionStorage.setItem('room', joinRoomSelection)
    } else {
      sessionStorage.setItem('room', roomFormValue)
    }
    navigate({ to: '/chat' })
  }

  useEffect(() => {
    if (user?.userId && roomName) {
      navigate({ to: '/chat', replace: true })
    }
  }, [])

  return (
    <LoginLayout>
      <Rooms
        rooms={rooms ?? []}
        selectionHandler={setJoinRoomSelection}
        selectedRoom={joinRoomSelection}
        isLoading={roomsLoading}
      ></Rooms>
      <LoginForm
        defaultUser={user?.userName}
        disableNewRoom={joinRoomSelection !== ''}
        login={login}
      ></LoginForm>
    </LoginLayout>
  )
}

export const loader = async () => {
  const user = getUser()
  return {
    user: user,
    roomName: sessionStorage.getItem('room'),
  }
}

type LoginLocationGenerics = MakeGenerics<{
  LoaderData: {
    user: Pick<User, 'userId' | 'userName'>
    roomName: string
  }
}>

export default Login

Let’s drill in piece by piece in the Login page and pick apart what’s going on.

First lets look at the bottom of the file at loader and LoginLocationGenerics.

We’re exporting this loader function for use in our routes.tsx file to load some data that we want ahead of page rendering time. Both the user and roomName return values are being grabbed from sessionStorage.

export const loader = async () => {
  const user = getUser()
  return {
    user: user,
    roomName: sessionStorage.getItem('room'),
  }
}

getUser() is just a helper method for getting user values from session storage.

client/lib/user.ts

//...

export const getUser = () => {
  const userId = sessionStorage.getItem('userId')
  const userName = sessionStorage.getItem('userName')
  return {
    userId,
    userName,
  }
}

//...

With this loader data we can define a generic type with MakeGenerics from React Location which gives us some type safety for the loader data which we’ll extract in our page - we’ll see that next.

type LoginLocationGenerics = MakeGenerics<{
  LoaderData: {
    user: Pick<User, 'userId' | 'userName'>
    roomName: string
  }
}>

We can use useMatch from React Location to get our loader data - and pass our generic type to it.

const {
  data: { user, roomName },
} = useMatch<LoginLocationGenerics>()

Our login page has both a form to create and join a new room, and also to select from existing rooms that have been created. To store an existing room selection we make use of a simple useState hook from React.

const [joinRoomSelection, setJoinRoomSelection] = useState<string>('')

To fetch the existing rooms we’ve created a custom useRoomsQuery hook.

const { data: rooms, isLoading: roomsLoading } = useRoomsQuery()

Under the hood for this we’re using useQuery from React Query and fetching our server’s /api/rooms endpoint with axios. This effectively loads all data for all rooms into the client query cache - freaking awesome!

client/lib/rooms.ts

//...

export const useRoomsQuery = () => {
  const query = useQuery({
    queryKey: ['select_rooms'],
    queryFn: (): Promise<Room[]> => axios.get(`/api/rooms`).then((response) => response.data),
  })
  return query
}

//...

There are a few scenarios for the Login page for which we may want or need the user to navigate to another route.

Whenever a user creates or joins an existing room, or if a user attempts to come to the login page while already being logged in and in an existing room - we need to direct them over to the Chat page.

To do this sort of navigation, we can use useNavigate from React Location - so here we instantiate it.

const navigate = useNavigate()

We need a function to handle when a user submits the login form, so we’ve created a login function to handle just that. We’re grabbing form values and also checking if the user has chosen an existing room to join - and finally navigation to the /chat route.

const login = (e: React.FormEvent<HTMLFormElement>) => {
  const userFormValue = e.target[0].value
  const roomFormValue = e.target[1].value
  const newUser = {
    userId: generateUserId(userFormValue),
    userName: userFormValue,
  }
  setUser({ id: newUser.userId, name: newUser.userName })
  if (joinRoomSelection !== '') {
    sessionStorage.setItem('room', joinRoomSelection)
  } else {
    sessionStorage.setItem('room', roomFormValue)
  }
  navigate({ to: '/chat' })
}

The second scenario for if a user is already logged in and in an existing room, but tries to navigate to the Login page, we need to direct them back to the Chat page.

We can do this with a useEffect that runs only once on the first render to do some validation and perform the navigate.

useEffect(() => {
  if (user?.userId && roomName) {
    navigate({ to: '/chat', replace: true })
  }
}, [])

Finally we return the UI components.

return (
  <LoginLayout>
    <Rooms
      rooms={rooms ?? []}
      selectionHandler={setJoinRoomSelection}
      selectedRoom={joinRoomSelection}
      isLoading={roomsLoading}
    ></Rooms>
    <LoginForm
      defaultUser={user?.userName}
      disableNewRoom={joinRoomSelection !== ''}
      login={login}
    ></LoginForm>
  </LoginLayout>
)

Let’s see what our Rooms UI component looks like.

This Rooms component is relatively straightforward, but one thing of note is a delay functionality we’ve added. When rooms data gets loaded, there’s a potential for that data to load fast - causing the loading spinner to flicker. To avoid this we’re intentionally delaying the rendering of the rooms and keeping the spinner on the page for a minimum of 1000 milliseconds.

client/components/rooms.tsx

import React, { useEffect, useState } from 'react'
import { Room } from '../../shared/interfaces/chat.interface'
import { Loading } from './loading'

export const Rooms = ({
  rooms,
  selectionHandler,
  selectedRoom,
  isLoading,
}: {
  rooms: Room[]
  selectionHandler: (roomName: string) => void
  selectedRoom?: string
  isLoading: boolean
}) => {
  const [isDelay, setIsDelay] = useState(true)
  useEffect(() => {
    const delayTimer = setTimeout(() => {
      setIsDelay(false)
    }, 1000)
    return () => {
      clearTimeout(delayTimer)
    }
  }, [])

  return (
    <div className="h-full w-full rounded-lg border border-slate-400 bg-gray-800 md:h-1/2">
      <div className="flex justify-between rounded-t-md border border-slate-400 bg-slate-400 p-2">
        <span>Join existing rooms</span>
        {selectedRoom && <button onClick={() => selectionHandler('')}>Clear</button>}
      </div>
      <div className="w-full">
        {!isLoading &&
          !isDelay &&
          rooms.map((room, index) => (
            <button
              key={index}
              className={
                selectedRoom === room.name
                  ? 'w-full bg-slate-900 p-2 text-left text-gray-400'
                  : ' w-full p-2 text-left text-gray-400'
              }
              onClick={() => selectionHandler(room.name)}
            >
              {room.name}
            </button>
          ))}
        {(isLoading || isDelay) && <Loading />}
      </div>
    </div>
  )
}

The LoginForm is almost as straightforward as it gets!

client/components/login.form.tsx

import React from 'react'

export const LoginForm = ({
  login,
  disableNewRoom,
  defaultUser,
}: {
  login: (e: React.FormEvent<HTMLFormElement>) => void
  disableNewRoom: boolean
  defaultUser?: string
}) => {
  return (
    <div className="h-full w-full py-2 md:px-2 md:py-0">
      <form
        onSubmit={(e) => {
          e.preventDefault()
          login(e)
        }}
        className="flex flex-col justify-center"
      >
        <input
          type="text"
          id="login"
          placeholder="Name"
          defaultValue={defaultUser && defaultUser}
          required={true}
          className="mb-2 h-12 rounded-md border border-slate-400 bg-gray-800 text-white placeholder-slate-400 focus:border-violet-500 focus:outline-none focus:ring-1 focus:ring-violet-500"
        ></input>
        <input
          type="text"
          id="room"
          disabled={disableNewRoom}
          placeholder="New room"
          className="mb-2 h-12 rounded-md border border-slate-400 bg-gray-800 text-white placeholder-slate-400 focus:border-violet-500 focus:outline-none focus:ring-1 focus:ring-violet-500 disabled:opacity-50"
        ></input>
        <button
          type="submit"
          className="flex h-12 w-full items-center justify-center rounded-md bg-violet-700 text-white"
        >
          Join
        </button>
      </form>
    </div>
  )
}

Here are some snapshots of the Login page!

Let’s move on to the really fun part of the client - the Chat page!

client/pages/chat.tsx

import React, { useState, useEffect } from 'react'
import { MakeGenerics, useMatch, useNavigate } from '@tanstack/react-location'
import { io, Socket } from 'socket.io-client'
import {
  User,
  Message,
  ServerToClientEvents,
  ClientToServerEvents,
} from '../../shared/interfaces/chat.interface'
import { Header } from '../components/header'
import { UserList } from '../components/list'
import { MessageForm } from '../components/message.form'
import { Messages } from '../components/messages'
import { ChatLayout } from '../layouts/chat.layout'
import { unsetRoom, useRoomQuery } from '../lib/room'
import { getUser } from '../lib/user'

const socket: Socket<ServerToClientEvents, ClientToServerEvents> = io({
  autoConnect: false,
})

function Chat() {
  const {
    data: { user, roomName },
  } = useMatch<ChatLocationGenerics>()

  const [isConnected, setIsConnected] = useState(socket.connected)
  const [messages, setMessages] = useState<Message[]>([])
  const [toggleUserList, setToggleUserList] = useState<boolean>(false)

  const { data: room } = useRoomQuery(roomName, isConnected)

  const navigate = useNavigate()

  useEffect(() => {
    if (!user || !roomName) {
      navigate({ to: '/', replace: true })
    } else {
      socket.on('connect', () => {
        socket.emit('join_room', {
          roomName,
          user: { socketId: socket.id, ...user },
        })
        setIsConnected(true)
      })

      socket.on('disconnect', () => {
        setIsConnected(false)
      })

      socket.on('chat', (e) => {
        setMessages((messages) => [e, ...messages])
      })

      socket.connect()
    }
    return () => {
      socket.off('connect')
      socket.off('disconnect')
      socket.off('chat')
    }
  }, [])

  const leaveRoom = () => {
    socket.disconnect()
    unsetRoom()
    navigate({ to: '/', replace: true })
  }

  const sendMessage = (message: string) => {
    if (user && socket && roomName) {
      socket.emit('chat', {
        user: {
          userId: user.userId,
          userName: user.userName,
          socketId: socket.id,
        },
        timeSent: new Date(Date.now()).toLocaleString('en-US'),
        message,
        roomName: roomName,
      })
    }
  }
  return (
    <>
      {user?.userId && roomName && room && (
        <ChatLayout>
          <Header
            isConnected={isConnected}
            users={room?.users ?? []}
            roomName={roomName}
            handleUsersClick={() => setToggleUserList((toggleUserList) => !toggleUserList)}
            handleLeaveRoom={() => leaveRoom()}
          ></Header>
          {toggleUserList ? (
            <UserList room={room}></UserList>
          ) : (
            <Messages user={user} messages={messages}></Messages>
          )}
          <MessageForm sendMessage={sendMessage}></MessageForm>
        </ChatLayout>
      )}
    </>
  )
}

export const loader = async () => {
  const user = getUser()
  return {
    user: user,
    roomName: sessionStorage.getItem('room'),
  }
}

type ChatLocationGenerics = MakeGenerics<{
  LoaderData: {
    user: Pick<User, 'userId' | 'userName'>
    roomName: string
  }
}>

export default Chat

Okay, the Chat page has quite a bit going on - but a couple of things like the loader, MakeGenerics, and the useMatch to load route data are all the same as the Login page so we’ll go over what’s new.

Since the last part of the series we’ve updated one part of instantiating our socket which is passing autoConnect: false. By disabling automatic connection - we are taking control over the when the connection happens. Why did we need this? Well there’s one scenario where if a user attempts to navigate to the Chat page while not being logged in or in a specific room, if automatic connection is true then a socket connection would occur when we don’t want it to. We first need to do some validation before connecting the socket.

const socket: Socket<ServerToClientEvents, ClientToServerEvents> = io({
  autoConnect: false,
})

We have some of the same useState hooks from the last series but also added one additional to store a flag for whether to display current users connected to the room.

const [isConnected, setIsConnected] = useState(socket.connected)
const [messages, setMessages] = useState<Message[]>([])
const [toggleUserList, setToggleUserList] = useState<boolean>(false)

We need data stored on the server about the room, so we had a custom useRoomQuery hook to fetch data from the server.

const { data: room } = useRoomQuery(roomName, isConnected)

The useRoomsQuery hook fetches data from axios and we use a couple of options to customize it’s behavior.

Since we’re primarily using this to fetch all users that are connected to the room, we want it to poll the server on an interval since people may be leaving or joining at any given time - in other words that data will go stale.

To handle polling, we pass refetchInterval: 60000 which will refetch data every 60 seconds.

Additionally we only want this query to run if the socket is connected so we pass enabled: isConnected as well.

client/lib/room.ts

export const useRoomQuery = (roomName, isConnected) => {
  const query = useQuery({
    queryKey: ['rooms', roomName],
    queryFn: (): Promise<Room> =>
      axios.get(`/api/rooms/${roomName}`).then((response) => response.data),
    refetchInterval: 60000,
    enabled: isConnected,
  })
  return query
}

Similar to the Login page, we have two scenarios where we want the user to navigate off of the page, so we need to instantiate useNavigate from React Location.

const navigate = useNavigate()

There are two important new things that we added to the useEffect on the Chat page.

Checking if user and roomName exists - and if either does not exist then we want to navigate back to the Login page route.
When the socket 'connect' event occurs, we want to emit a 'join_room' event to the server which will add us to the new or existing room.

useEffect(() => {
  if (!user || !roomName) {
    navigate({ to: '/', replace: true })
  } else {
    socket.on('connect', () => {
      socket.emit('join_room', {
        roomName,
        user: { socketId: socket.id, ...user },
      })
      setIsConnected(true)
    })

    socket.on('disconnect', () => {
      setIsConnected(false)
    })

    socket.on('chat', (e) => {
      setMessages((messages) => [e, ...messages])
    })

    socket.connect()
  }
  return () => {
    socket.off('connect')
    socket.off('disconnect')
    socket.off('chat')
  }
}, [])

We also added the ability for users to leave rooms.

const leaveRoom = () => {
  socket.disconnect()
  unsetRoom()
  navigate({ to: '/', replace: true })
}

The unsetRoom function is pretty simple. It just removes the current room from session storage.

client/lib/room.ts

export const unsetRoom = () => {
  sessionStorage.removeItem('room')
}

The sendMessage also hasn’t changed much and just handles sending messages to the room. We did add roomName as another part of the chat message data so the server knows which room to send the message in.

const sendMessage = (message: string) => {
  if (user && socket && roomName) {
    socket.emit('chat', {
      user: {
        userId: user.userId,
        userName: user.userName,
        socketId: socket.id,
      },
      timeSent: new Date(Date.now()).toLocaleString('en-US'),
      message,
      roomName: roomName,
    })
  }
}

Finally we return UI components.

return (
  <>
    {user?.userId && roomName && room && (
      <ChatLayout>
        <Header
          isConnected={isConnected}
          users={room?.users ?? []}
          roomName={roomName}
          handleUsersClick={() => setToggleUserList((toggleUserList) => !toggleUserList)}
          handleLeaveRoom={() => leaveRoom()}
        ></Header>
        {toggleUserList ? (
          <UserList room={room}></UserList>
        ) : (
          <Messages user={user} messages={messages}></Messages>
        )}
        <MessageForm sendMessage={sendMessage}></MessageForm>
      </ChatLayout>
    )}
  </>
)

The Header component is very straightforward. We have the following features in the Header:

A 🟢 or 🔴 circle indicating the connected state of the user’s socket.
The name of the room the user is connected to.
A button 👨‍💻 that will toggle the display between messages and connected users in the room.
A Leave button which directs users back to the Login page.

client/components/header.tsx

import React from 'react'
import { User } from '../../shared/interfaces/chat.interface'

export const Header = ({
  isConnected,
  users,
  handleUsersClick,
  handleLeaveRoom,
  roomName,
}: {
  isConnected: boolean
  users: User[]
  handleUsersClick: () => void
  handleLeaveRoom: () => void
  roomName: string
}) => {
  return (
    <header className="flex h-1/6 flex-col pt-12">
      <div className="flex justify-between">
        <div className="flex h-8 items-center">
          <span className="ml-1">{isConnected ? '🟢' : '🔴'}</span>
          <span className="px-2 text-3xl text-white">{'/'}</span>
          <span className=" text-white">{roomName}</span>
        </div>
        <div className="flex">
          <button
            onClick={() => handleUsersClick()}
            className="ml-1 flex h-8 items-center rounded-xl bg-gray-800 px-4"
          >
            <span className="mr-1 text-lg text-white">{'👨‍💻'}</span>
            <span className="ml-1 text-white">{users.length}</span>
          </button>
          <button
            onClick={() => handleLeaveRoom()}
            className="ml-1 flex h-8 items-center rounded-xl bg-gray-800 px-4"
          >
            <span className="mr-1 text-white">{'Leave'}</span>
          </button>
        </div>
      </div>
    </header>
  )
}

The Messages component displays all messages in the room since the user has been connected. There is no persistent storage yet but we may be adding this in later parts of this series.

We have the following features in the Messages component:

Display messages with the latest message at the bottom.
Allow scrolling on the y-axis.
Display the user and the time sent.
Style the current user’s messages in a light gray and a left margin, and the opposite for all other users.

client/components/messages.tsx

import React from 'react'
import { Message, User } from '../../shared/interfaces/chat.interface'

const determineMessageStyle = (user: Pick<User, 'userId' | 'userName'>, messageUserId: string) => {
  if (user && messageUserId === user.userId) {
    return {
      message: 'bg-slate-500 p-4 ml-24 mb-4 rounded break-words',
      sender: 'ml-24 pl-4',
    }
  } else {
    return {
      message: 'bg-slate-800 p-4 mr-24 mb-4 rounded break-words',
      sender: 'mr-24 pl-4',
    }
  }
}

export const Messages = ({
  user,
  messages,
}: {
  user: Pick<User, 'userId' | 'userName'>
  messages: Message[]
}) => {
  return (
    <div className="flex h-4/6 w-full flex-col-reverse overflow-y-scroll">
      {messages?.map((message, index) => {
        return (
          <div key={index}>
            <div className={determineMessageStyle(user, message.user.userId).sender}>
              <span className="text-sm text-gray-400">{message.user.userName}</span>
              <span className="text-sm text-gray-400">{' ' + '•' + ' '}</span>
              <span className="text-sm text-gray-400">{message.timeSent}</span>
            </div>
            <div className={determineMessageStyle(user, message.user.userId).message}>
              <p className="text-white">{message.message}</p>
            </div>
          </div>
        )
      })}
    </div>
  )
}

The UserList component has the following features:

Display all current users in the room.
Display a 👑 next to the host of the room.

client/components/list.tsx

import React from 'react'
import { Room } from '../../shared/interfaces/chat.interface'

export const UserList = ({ room }: { room: Room }) => {
  return (
    <div className="flex h-4/6 w-full flex-col-reverse overflow-y-scroll">
      {room.users.map((user, index) => {
        return (
          <div key={index} className="mb-4 flex rounded px-4 py-2">
            <p className="text-white">{user.userName}</p>
            {room.host.userId === user.userId && <span className="ml-2">{'👑'}</span>}
          </div>
        )
      })}
    </div>
  )
}

Lastly, and importantly we have the MessageForm component where users can input new messages.

The MessageForm component has the following features:

Allow users to submit a message via either a onKeyDown event when the user presses the 'Enter' key or an onClick event on the send button.
Once a message is sent, clear the form value.

client/components/message.form.tsx

import React, { useRef } from 'react'

export const MessageForm = ({ sendMessage }: { sendMessage: (message: string) => void }) => {
  const textAreaRef = useRef<HTMLTextAreaElement>(null)

  const submit = (e) => {
    e.preventDefault()
    const value = textAreaRef?.current?.value
    if (value) {
      sendMessage(value)
      textAreaRef.current.value = ''
    }
  }

  const handleKeyDown = (e) => {
    if (e.key === 'Enter') {
      submit(e)
    }
  }

  return (
    <div className="flex h-1/6 items-center">
      <form className="flex w-full appearance-none rounded-md bg-gray-800 outline-none focus:outline-none">
        <textarea
          ref={textAreaRef}
          onKeyDown={(e) => handleKeyDown(e)}
          id="minput"
          placeholder="Message"
          className="mb-2 max-h-16 flex-grow appearance-none rounded-md border-none bg-gray-800 text-white placeholder-slate-400 focus:outline-none focus:ring-transparent"
        ></textarea>
        <button onClick={(e) => submit(e)} className="self-end p-2 text-slate-400">
          <svg
            xmlns="http://www.w3.org/2000/svg"
            fill="none"
            viewBox="0 0 24 24"
            strokeWidth={1.5}
            stroke="currentColor"
            className="h-4 w-4 bg-gray-800"
          >
            <path
              strokeLinecap="round"
              strokeLinejoin="round"
              d="M6 12L3.269 3.126A59.768 59.768 0 0121.485 12 59.77 59.77 0 013.27 20.876L5.999 12zm0 0h7.5"
            />
          </svg>
        </button>
      </form>
    </div>
  )
}

Here are the snapshots of the Chat page now!

That's all for this part of the series.

Nest JS Websockets - Basics

Delightful Engineering — Mon, 05 Dec 2022 04:36:12 +0000

Welcome to part 1, the Basics of building a chat application with Nest JS and React.

This series will attempt to touch on as many features of Nest's websockets integration as well as socket.io in general.

Download the repository at this commit to follow along.

Background

Nest JS is a framework that provides a level of abstraction above common Node frameworks like Express, Fastify, and importantly for this use case - Socket.io. Nest is particularly opinionated about architecture which on one hand means you're "pigeon holed", but on the other hand they've made decisions for you which can allow you to focus on building your application.

React doesn't need much of an introduction as it's such a popular framework. Nevertheless React is a framework for building interactive user interfaces with a ton of broad support across the internet. Given any roadblocks you might run into, someone has most likely posted a solution somewhere or is likely to help you out.

Socket.io is a Node framework around websockets which allow for two-way communication between clients (our React client) and servers (our Nest server) - otherwise known as full-duplex systems.

Design

For a basic first iteration of the application, we'll shoot for this kind of overall design:

We want our clients to be able to emit messages to our server via a form input.
We want our server to subscribe to messages coming in from clients, and emit or broadcast them out to all connected clients.

File Structure

For this project we're developing both the client and the server in the same repository. The Nest server is also serving the React client so it makes since to have them together.

The core structure at this stage looks like this:

public: static files like images and importantly index.html which gets served initially on page render.
src
- client: react client source files.
- server: nest server source files.
- shared: source files shared by client and server - mostly share Typescript types for full-stack type safety.
...configuration and miscellaneous files: the rest of the files in the root directory are files such as .gitignore, package.json, webpack.config.client.js for bundling our react client, tailwind.config.js for our css library configuration, etc.

The Server

At this stage our server is quite small - only 4 files.

chat
- chat.gateway.ts
- chat.module.ts
app.module.ts
main.ts

Let's take a look at each of the files one by one and see what's happening in them.

import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import { NestExpressApplication } from '@nestjs/platform-express';

async function bootstrap() {
  const app = await NestFactory.create<NestExpressApplication>(AppModule);
  await app.listen(3000);

  // Gracefully shutdown the server.
  app.enableShutdownHooks();
}
bootstrap();

The main.ts file is our server's entrypoint. We the server boots up, this is the first file that's executed. As was mentioned before, Nest provides a level of abstraction above Express and Fastify - and we are going with the Exress flavor here as seen when we create the server with NestExpressApplication from the '@nestjs/platform-express' package. Our server runs on port 3000 and we make use of a handy app.enableShutdownHooks() to shut down our server gracefully in the event that the running process is killed - read about shutdown hooks here.

import { Module } from '@nestjs/common';
import { ChatModule } from './chat/chat.module';
import { ServeStaticModule } from '@nestjs/serve-static';
import { join } from 'path';

@Module({
  imports: [
    ChatModule,
    ServeStaticModule.forRoot({
      rootPath: join(__dirname, '..', '..', '..', 'dist', 'client'),
    }),
  ],
})
export class AppModule {}

Nest employs the concept of modules as part of it's core architecture. Our app.module.ts file contains our root module which is the starting point that Nest uses to build the application graph. Any other modules in the server would be children of the application module and should be imported into their direct parents. In this case, we have a ChatModule that we'll take a look at shortly, but here we are adding it to our application module's imports: [] array.

We want our Nest server to also serve our client React application. To do that we can make use of the ServeStaticModule from '@nestjs/serve-static'. It takes a rootPath path to the bundle of our client that gets produced by webpack.

import {
  MessageBody,
  SubscribeMessage,
  WebSocketGateway,
  WebSocketServer,
} from '@nestjs/websockets';
import { Logger } from '@nestjs/common';
import {
  ServerToClientEvents,
  ClientToServerEvents,
  Message,
} from '../../shared/interfaces/chat.interface';
import { Server } from 'socket.io';

@WebSocketGateway({
  cors: {
    origin: '*',
  },
})
export class ChatGateway {
  @WebSocketServer() server: Server = new Server<
    ServerToClientEvents,
    ClientToServerEvents
  >();
  private logger = new Logger('ChatGateway');

  @SubscribeMessage('chat')
  async handleEvent(
    @MessageBody()
    payload: Message,
  ): Promise<Message> {
    this.logger.log(payload);
    this.server.emit('chat', payload); // broadcast messages
    return payload;
  }
}

Here is the really interesting part of the server. This file is the websocket gateway for our chat application - see Gateways. All we need to do to create a gateway is importing and using the WebsocketGateway class and pass it some configuration options - in our case we're setting cors to be wide open to all origins by using '*'. This is not considered safe and should only be open to trusted origins - we will address this later in future series parts.

Nest supports 2 websocket platforms - socket.io and ws. We're going to be using socket.io.

To get access to the underlying socket.io server, we can use the WebSocketServer class to instantiate a server with a type of socket.io Server. One extremely cool thing we can do here in terms of Typescript is passing in event types ServerToClientEvents and ClientToServerEvents. We will touch on these again when we go over our shared directory.

We want to log chat messages coming into the server from clients so we also instantiate a logger with Nest's Logger class.

We also need to subscribe to incoming messages from clients. To do this we can use the SubscribeMessage class which takes the name of the messages chat. This requires passing a function to handle the incoming message which takes a MessageBody payload of type Message which we also get from our shared directory since both client and server will want to know the message type. We then log the message with this.logger.log(payload) and then emit the message by invoking the socket.io server's emit method with this.server.emit('chat', payload). This will broadcast the message to all clients that have subscribed to chat messages. Then we return the payload.

import { Module } from '@nestjs/common';
import { ChatGateway } from './chat.gateway';

@Module({
  providers: [ChatGateway],
})
export class ChatModule {}

The chat.module.ts file is as simple as it gets. Nest treats WebSocketGateways as providers which like modules are part of it's core architecture. We pass our ChatGateway into providers: [].

The Client

We're going to try to focus on the websockets aspect of the React client and skip over some things like component styling, webpack configuration, postcss configuration, etc. One thing you'll also notice is we don't have any client-side routing - which we may end up adding in later series parts. But for now we're sticking to the basics.

Our server directory looks like this:

components: UI components.
layouts: UI page layouts.
styles: core css style sheet.
app.tsx: core application file.
index.tsx: client application entry point.

Let's just focus on the app.tsx file which contains the core logic of the frontend.

import React, { useState, useEffect } from 'react';
import { io, Socket } from 'socket.io-client';
import {
  User,
  Message,
  ServerToClientEvents,
  ClientToServerEvents,
} from '../shared/interfaces/chat.interface';
import { Header } from './components/header';
import { LoginForm } from './components/login.form';
import { MessageForm } from './components/message.form';
import { Messages } from './components/messages';
import { ChatLayout } from './layouts/chat.layout';
import { LoginLayout } from './layouts/login.layout';

const socket: Socket<ServerToClientEvents, ClientToServerEvents> = io();

function App() {
  const [isConnected, setIsConnected] = useState(socket.connected);
  const [messages, setMessages] = useState<Message[]>([]);
  const [user, setUser] = useState<User>();

  useEffect(() => {
    const currentUser = JSON.parse(sessionStorage.getItem('user') ?? '{}');
    if (currentUser.userId) {
      setUser(currentUser);
    }

    socket.on('connect', () => {
      setIsConnected(true);
    });

    socket.on('disconnect', () => {
      setIsConnected(false);
    });

    socket.on('chat', (e) => {
      setMessages((messages) => [e, ...messages]);
    });

    return () => {
      socket.off('connect');
      socket.off('disconnect');
      socket.off('chat');
    };
  }, []);

  const login = (e: React.FormEvent<HTMLFormElement>) => {
    const formValue = e.target[0].value;
    const newUser = {
      userId: Date.now().toLocaleString().concat(formValue),
      userName: formValue,
    };
    sessionStorage.setItem('user', JSON.stringify(newUser));
    setUser(newUser);
  };

  const sendMessage = (e: React.FormEvent<HTMLFormElement>) => {
    if (user) {
      socket.emit('chat', {
        user: {
          userId: user.userId,
          userName: user.userName,
        },
        timeSent: new Date(Date.now()).toLocaleString('en-US'),
        message: e.target[0].value,
      });
    }
  };
  return (
    <>
      {user && user.userId ? (
        <ChatLayout>
          <Header user={user} isConnected={isConnected}></Header>
          <Messages user={user} messages={messages}></Messages>
          <MessageForm sendMessage={sendMessage}></MessageForm>
        </ChatLayout>
      ) : (
        <LoginLayout>
          <LoginForm login={login}></LoginForm>
        </LoginLayout>
      )}
    </>
  );
}

export default App;

Since we're using socket.io for our server websocket platform, we're also going to use another package that they ship called 'socket.io-client'.

After we import all the things we need at the top of the file, the first important thing we do is create a new socket instance with the io() function that we import from the package.

Like in our chat gatway on the server, we're passing in our shared types ServerToClientEvents and ClientToServerEvents.

We have 3 useState hooks where we're storing some state in the client.

// We want to store the client's current state of the connection.
const [isConnected, setIsConnected] = useState(socket.connected)
// We want to store a list of messages that have been emitted from the server.
const [messages, setMessages] = useState<Message[]>([])
// We want to store the current user of the client.
const [user, setUser] = useState<User>()

We also have one useEffect hook where we want to perform some side effects. We're going to leverage a browser api sessionStorage to temporarily store the currently "logged in" user in the browser. There are some nuances between sessionStorage and localStorage. For this version of the application we're going to use sessionStorage so that we can log in a new user per each browser tab that we open for easy demonstration purposes. We're also not doing any authentication at the moment so we need not burden ourselves with security at this stage - sessionStorage is just fine for now.

useEffect(() => {
  // Grab the currently "logged in" user from sessionStorage
  const currentUser = JSON.parse(sessionStorage.getItem('user') ?? '{}')
  // If userId exists, we know there is a "logged in" user.
  if (currentUser.userId) {
    // So we set the current user into state.
    setUser(currentUser)
  }

  // We go ahead and setup an event listener for a 'connect' event.
  socket.on('connect', () => {
    // Set connect state accordingly.
    setIsConnected(true)
  })

  // We go ahead and setup an event listener for a 'disconnect' event.
  socket.on('disconnect', () => {
    // Set connect state accordingly.
    setIsConnected(false)
  })

  // We go ahead and setup an event listener for a 'disconnect' event.
  socket.on('chat', (e) => {
    // Set connect state accordingly.
    setMessages((messages) => [e, ...messages])
  })

  // We define a cleanup procedure to remove event listeners.
  return () => {
    socket.off('connect')
    socket.off('disconnect')
    socket.off('chat')
  }
}, [])

We define a login function that we can execute when a new user submits their name in a login form.

const login = (e: React.FormEvent<HTMLFormElement>) => {
  // Extract user input from the form.
  const formValue = e.target[0].value
  // Construct a new user object with user input formValue.
  const newUser = {
    // Unique userId by concating current date and the user's name.
    // The edge case here would be if two users of the same name logged
    // in at the exact same millisecond in time.
    // We can solve this edge case later with authentication.
    // Ex) "1,668,125,065,218anna"
    userId: Date.now().toLocaleString().concat(formValue),
    userName: formValue,
  }
  // Set new user to session storage.
  sessionStorage.setItem('user', JSON.stringify(newUser))
  // Set new user to state.
  setUser(newUser)
}

We define another function sendMessage that we can execute when a user submits a message in a message form.

const sendMessage = (e: React.FormEvent<HTMLFormElement>) => {
  // There must be a user set in state to send a message.
  if (user) {
    // Given a user is "logged in", emit a 'chat' message to the server
    // with the current user's information, the time of the message (timeSent),
    // and the value of the user input in the message form.
    socket.emit('chat', {
      user: {
        userId: user.userId,
        userName: user.userName,
      },
      timeSent: new Date(Date.now()).toLocaleString('en-US'),
      message: e.target[0].value,
    })
  }
}

Here is a sample Message object:

{
  "user": {
    "userId": "1,667,963,682,100anna",
    "userName": "anna"
  },
  "timeSent": "11/10/2022, 8:41:35 PM",
  "message": "🤨"
}

Finally we return some tsx/jsx elements from our app function. We won't look into each and every React component for the sake of brevity but it's encouraged to browse through the components and layout directory to see what's happening.

return (
  <>
    {/* If a user is "logged in" (set in state), render chat elements. */}
    {user && user.userId ? (
      <ChatLayout>
        <Header user={user} isConnected={isConnected}></Header>
        <Messages user={user} messages={messages}></Messages>
        <MessageForm sendMessage={sendMessage}></MessageForm>
      </ChatLayout>
    ) : (
      {/* If a user is not "logged in" (set in state), render login elements. */}
      <LoginLayout>
        <LoginForm login={login}></LoginForm>
      </LoginLayout>
    )}
  </>
)

That's it for our client for now! Here's what we're left with:

Login UI

Chat UI

Shared Types

Before we wrap this up we should touch on one of the key features of this setup - shared types.

Checkout the Typescript docs on socket.io.

In our shared directory we have a common chat.interface.ts file.

export interface User {
  userId: string;
  userName: string;
}

export interface Message {
  user: User;
  timeSent: string;
  message: string;
}

// Interface for when server emits events to clients.
export interface ServerToClientEvents {
  chat: (e: Message) => void;
}

// Interface for when clients emit events to the server.
export interface ClientToServerEvents {
  chat: (e: Message) => void;
}

If you can remember before, we're using these event interfaces in both the client and the server.

// ...

@WebSocketServer() server: Server = new Server<
    ServerToClientEvents,
    ClientToServerEvents
  >();

// ...

async handleEvent(
  @MessageBody()
  payload: Message,
): Promise<Message> {
  this.logger.log(payload);
  this.server.emit('chat', payload); // <--- ServerToClientEvent
  return payload;
}

// ...

// ...

const socket: Socket<ServerToClientEvents, ClientToServerEvents> = io();

// ...

socket.emit('chat', { // <--- ClientToServerEvent
  user: {
    userId: user.userId,
    userName: user.userName,
  },
  timeSent: new Date(Date.now()).toLocaleString('en-US'),
  message: e.target[0].value,
});

// ...

With these interfaces, we have end-to-end type safety which provides us with a strong layer of assurance that our server and clients will receive the correct data! Super cool.

That's all for this part of the series! Go check out the code and more specifically the UI components that we didn't touch on here like the Messages, MessageForm, and the LoginForm and take a look at some of the tailwind css that we've done for styling them.

Download the repository at this commit to follow along.

Lerna and Turborepo with Gitlab CI/CD

Delightful Engineering — Sun, 25 Sep 2022 02:43:44 +0000

In this post we're going to cover a simple approach to a monorepo setup with a fully automated CI/CD workflow with Lerna, Turborepo, and Gitlab CI/CD.

Here is the project link to follow along and use for your own monorepo projects.

Tools

This workflow includes the following tools.

Lerna: for versioning and publishing packages.
Turborepo: for tasking running.
Gitlab CI/CD: for our CI/CD server.
Husky for Git hooks.
Commitizen: for its interactive commit cli and changelog adapter. We will tie this into one of our Git hooks through husky - they work nicely together.

Typescript and Jest Configurations

One helpful way to manage Typescript and Jest configurations in a monorepo is to define core configurations in the root of the repository, and extend from the core configurations inside of packages to suit individual package needs.

The tsconfig.json file in the root of the repository has some basic configurations.

{
  "compilerOptions": {
    "module": "commonjs",
    "moduleResolution": "node",
    "declaration": true,
    "noImplicitAny": true,
    "removeComments": true,
    "target": "es6",
    "sourceMap": true,
    "strict": true
  },
  "exclude": ["node_modules"]
}

The packages also have their own tsconfig.json files that make use of the "extends" option that will inherit from the root, and any new settings will override root configuration values.

{
  "extends": "../../tsconfig.json",
  "compilerOptions": {
    "outDir": "./lib"
  },
  "include": ["./src"]
}

Similarly for jest we have a root configuration that is extended and overridable in packages.

import type { Config } from "jest";

const config: Config = {
  verbose: true,
  preset: "ts-jest",
  collectCoverage: true,
};

export default config;

import type { Config } from "jest";
import { default as rootConfig } from "../../jest.config";

const config: Config = {
  ...rootConfig,
};

export default config;

Git, Versioning and Publishing Strategy

Let's break down what our strategy is in the diagram above. In this diagram, P1 and P2 stand for package-one and package-two. Since we're adhering to a trunk-based branching strategy, we have one main branch and as short-lived as possible working branches.

The main branch is where all stable versions of packages will be versioned and published.

Any commits to branches named anything other than main (working branches) will publish canary versions of packages if they have related changes in the commits. The git sha is attached to the version to avoid collisions. This gives us "nightly builds" on working branches so that work in progress can be quickly tested by anyone.

# example from lerna docs

lerna publish --canary
# 1.0.0 => 1.0.1-alpha.0+${SHA} of packages changed since the previous commit
# a subsequent canary publish will yield 1.0.1-alpha.1+${SHA}, etc

# this script we use in our root package.json

"publish:canary": "lerna publish --canary --no-git-tag-version --no-push --yes"

Any commits git commit to working branches are met with a Husky prepare-commit-msg hook that triggers Commitizen's interactive CLI that will prompt you with questions to construct a commit message based on Angular's commit message conventions. The hook also skips if running in CI. Gitlab CI/CD has a predefined variable CI that we can use to identify this.

#!/usr/bin/env sh
. "$(dirname -- "$0")/_/husky.sh"

[ -n "$CI" ] && exit 0

exec < /dev/tty && npx cz --hook || true

Keeping commits very specific and short is helpful and important for a number of reasons. Depending on what kind of commit you select in the interactive CLI will determine which stable version of the package ends up getting updated when merging to main branch. Commit messages will also be added to CHANGELOG.md files which are ideal to keep clean for keeping track of and reporting changes to consumers of your packages.

When merging back to the main branch, the pipeline will run lerna version with some additional arguments which will create and push tags for changed packages as well as update, commit, and push version bumps in their package.json files and CHANGELOG.md files. In order for the pipeline to push changes back to the repository, there are some CI/CD specific things we will need to do which are covered in the Gitlab CI/CD Strategy section.

"version:stable": "lerna version --yes"

We also have some settings in lerna.json for the version command that only allows versioning on the main branch and an adherance to conventional commits.

"command": {
  "version": {
    "allowBranch": "main",
    "conventionalCommits": true
  }
},

Since the first versioning pipeline that runs on a merge to main produces a commit and push, another pipeline is triggered which will use lerna publish with additional arguments. The from-git argument publishes packages tagged in the current commit from our versioning pipeline.

"publish:stable": "lerna publish from-git --yes"

Gitlab CI/CD Strategy

While we covered some CI/CD in the Git, Versioning and Publishing section, this section will focus more on tokens, Gitlab repository settings, and the .gitlab-ci.yml file. For this section you'll need to have access to your Gitlab repository settings - if it's your company Gitlab you may not have maintainer access.

Tokens

For this strategy we're going to need two secret tokens:

$NPM_TOKEN: to publish packages to the NPM registry or this could be your companies internal Nexus repository. In your npmjs account you can generate a new token in Access Tokens -> Generate New Token and give it an Automation type and make sure to copy the secret value. Now in Gitlab, in Settings -> CI/CD -> Variables add a new variable with key of NPM_TOKEN and value of your copied secret. Make sure to check the Masked option so that your token is masked in job logs. We don't want this token to only be exposed to protected branches, otherwise our canary publish on working branches will not work.
GL_TOKEN: our versioning pipeline on the main branch pushes tags and a commit with package version bumps and changelog updates. In order for our CI server to be able to do this, we need to give it permissions to do so. For free personal Gitlab accounts you can create a personal access token otherwise if project access tokens are available create one with api, read_repository and write_repository permissions. Now as we did the the $NPM_TOKEN, add this as another CI/CD variable called $GL_TOKEN. If it's a project access token you'll need to assign it a role that has access to push to the main branch. Also, in case you wanted to have Lerna create Gitlab releases, your token would need registry read and write access - but that is out of the scope of this strategy.

YAML File

Let's pick apart our project's .gitlab-ci.yml file.

Firstly, we have 1 stage in our pipeline that will always run just 1 job. It may be helpful to minimize the number of stages and jobs in your pipelines to keep complexity low - this is a personal or team preference.

stages:
  - build

Next we have a .prepare configuration that we can extend in all of our jobs, since we will need to do this preparation before-hand. This is a cleaner way to reuse logic rather than have the same preparation script code in every job.

.prepare:
  before_script:
    - git config user.email $GITLAB_USER_EMAIL
    - git config user.name $GITLAB_USER_NAME
    - git remote set-url origin
      "https://gitlab-ci-token:$GL_TOKEN@$CI_SERVER_HOST/$CI_PROJECT_PATH.git"
    - echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" > .npmrc
    - git checkout $CI_COMMIT_BRANCH

We're specifying these to be run as before_script which will run before our jobs' primary scripts. Since Lerna will be performing Git operations during versioning and publishing, we set git config user.email $GITLAB_USER_EMAIL and git config user.name $GITLAB_USER_NAME. The $GITLAB_USER_EMAIL and $GITLAB_USER_NAME variables are predefined variables that are the email and name of the user who started the job.

Next we set the job token to assume the access of the $GL_TOKEN variable that we generated as either a personal or project access token with elevated repository permissions. The $CI_SERVER_HOST and $CI_PROJECT_PATH variables are also predefined variables that point to the url of the repository itself. See CI/CD job token. With this set, our release job will be able to push commits and tags back to the repository.

Next we're writing to a .npmrc file that we're wanting to publish to the public npm registry registry.npmjs.org and passing our $NPM_TOKEN variable as the authentication token so that we can publish packages. This could also be your company's internal Nexus repository in which case the registry would be different.

Now since Gitlab CI/CD jobs by default check out the repository at the commit that triggered the job in a DETACHED HEAD state, we need to actually checkout the branch itself so we run git checkout $CI_COMMIT_BRANCH, otherwise Lerna will throw an error.

Our first job in our YAML file build-publish-canary will only running on working branches and will publish our per commit nightly builds. It looks like this:

# CANARY PUBLISHING ON WORKING BRANCHES
build-publish-canary:
  stage: build
  extends:
    - .prepare
  script:
    - yarn install --frozen-lockfile
    - yarn build
    - yarn test
    - yarn publish:canary
  rules:
    - if:
        $CI_COMMIT_BRANCH != "main" && $CI_PIPELINE_SOURCE !=
        'merge_request_event' && $CI_COMMIT_TITLE != "Publish"

We're first making this job part of our 1 build stage. Then we're extending that .prepare configuration which will run its scripts before the script we have defined in this job.

The first part of our script is yarn install --frozen-lockfile which will install our dependencies using the exact state of our yarn.lock file that we have checked into the commit that triggered this job. This is generally a good practice for CI so that our lock file doesn't change when in CI. Then we run the build script defined in our root package.json file. This is where turbo comes in to run tasks. The build script looks like this:

"build": "turbo run build",

Turbo will run all of the build scripts defined in all of our monorepo's packages.

We then run the test script also defined in our root package.json in which turbo is also running our test tasks.

"test": "turbo run test",

Finally we run our publish:canary script which publishes to npm a unique package based on our commit.

An important part of this job are the rules that we have set.

$CI_COMMIT_BRANCH != main: ensures we're only running this job on branches that are not the main trunk branch.
$CI_PIPELINE_SOURCE != 'merge_request_event': ensures we're not running this job on merge requests.
$CI_COMMIT_TITLE != Publish: ensures that we do not run this job after Lerna has produced a commit with the title Publish which is what lerna version will do as we see in the next job.

Our second job build-version-stable will run on merges to main branch or on direct commit pushes to main branch. It looks like this:

# STABLE VERSIONING ON MAIN BRANCH
build-version-stable:
  stage: build
  extends:
    - .prepare
  script:
    - yarn install --frozen-lockfile
    - yarn build
    - yarn test
    - yarn version:stable
  rules:
    - if: $CI_COMMIT_BRANCH == "main" && $CI_COMMIT_TITLE != "Publish"

We'll skip what's already been explained in the previous job straight to our version:stablescript which we are running after testing. In the Git, Versioning and Publishing we covered that version:stable is going to create and push tags, update package.json versions and changelogs in our packages which will push a commit back to the main branch. That will then trigger our final job that we have defined.

# STABLE PUBLISHING ON MAIN BRANCH
build-publish-stable:
  stage: build
  extends:
    - .prepare
  script:
    - yarn install --frozen-lockfile
    - yarn build
    - yarn test
    - yarn publish:stable
  rules:
    - if: $CI_COMMIT_BRANCH == "main" && $CI_COMMIT_TITLE == "Publish"

This job will finally publish the new versions of our packages! 🥳

Tips

Don't squash commits on merging feature branches to main branch, the changelog will not reflect the changes that were made. It could be helpful to apply a constraint in Gitlab Settings -> Merge Requests -> Squash commits when merging and do not allow this.
When updating multiple packages at once on a feature branch, if you want canary publishes for all updated packages, you will need to push committed changes related to each package separately as canary publishing will refer to most recent commit on a feature branch to determine which packages to publish.
Avoid long running working branches. Commit and merge changes back to main as often as possible.
Work with two branches only, main and working branches. If bugs are found, simply use a working branch and merge fixes to main.
Communicate with your team on the value of Semantic Versioning, how to use Commitizen, and to be descriptive and intentional about commits.
If you're working with a team and certain members are experts or lead maintainers of specific packages, using Gitlab's Code Owners feature can be helpful for specifying specific maintainers who can approve merge requests for certain packages. This can be really helpful for splitting up responsibility. Code Owners requires Gitlab premium.

Node JS File Directory Search Algorithm

Delightful Engineering — Sun, 11 Sep 2022 00:57:51 +0000

In this post we're going to write a file system search algorithm in Typescript for Node-based programs with the following criteria:

A file tree of any size.
A given directory name to search for passed as an argument.
Always start the search from process.cwd() by default.
Return the first found occurence.
Return value should be an array with strings of the path to the found directory or empty array if not found.

Get started

First thing to do is to initialize a new Node JS project. For this we will use the Yarn package manager.

yarn init

# You can go ahead and just press enter through each option.

Next thing to do is add a few development dependencies so that we can run our program.

yarn add -D @types/node ts-node typescript

We can also go ahead and add a script to our package.json that we will use to run the program.

{
  "name": "file-tree-search",
  "scripts": {
    "fts": "ts-node src/index.ts"
  },
  "devDependencies": {
    "@types/node": "^18.7.16",
    "ts-node": "^10.9.1",
    "typescript": "^4.8.3"
  },
  "license": "MIT"
}

Next we'll create a src directory and some files for our program, and also a test directory will some directories and empty files that we can test as a file tree in our search algorithm.

# Create directories
mkdir src test test/hello test/world

# Create files
touch src/index.ts src/search.ts test/random.txt test/hello/another.txt test/world/pizza.txt

Create the search algorithm

Now we can write our search algorithm in search.ts.

import { readdirSync, statSync } from "fs";
import { join } from "path";

export const findDirectory = (
  searchName: string,
  startDirectory = [process.cwd()]
): string => {
  const scanStartDir = readdirSync(join(...startDirectory));
  const filteredScan = scanStartDir.filter((dir) =>
    isDir(join(...[...startDirectory, dir]))
  );
  if (filteredScan.includes(searchName)) {
    return join(...[...startDirectory, searchName]);
  } else if (filteredScan.length === 0) {
    return "";
  } else {
    let subScanFound = "";
    filteredScan.every((dir) => {
      const result = findDirectory(searchName, [...startDirectory, dir]);
      if (result.length > 0) {
        subScanFound = result;
        return false;
      }
      return true;
    });
    return subScanFound;
  }
};

const isDir = (path: string): boolean => {
  try {
    return statSync(path).isDirectory();
  } catch (e) {
    return false;
  }
};

Search algorithm explained

First we need functions provided by the standard Node library to interact with the file system. We'll import readdirSync to retrieve which files belong to a directory and statSync to get information about an individual file. When working with file systems, the "path" library provides an operating system agnostic way to deal with file paths. Paths are different on Windows vs. MacOS for example so we want our program to work with an type of file path structure. For joining strings together to create a path we will also import join from "path".

The two arguments to our findDirectory search function are:

searchName: a string of a directory name to search for.
startDirectory: a directory to start the search from which will go down the tree, we pass [process.cwd()] as the default will get the current working directory of the running Node process. The user can pass something else if they want (maybe searching from __dirname is useful for the user of our function). We're representing this as an array of strings so that we can later use to construct paths.

The first thing we want to do is read the files in the passed startDirectory and then filter those results to remove basic files from the results. Since we're searching for a directory, this is how we're narrowing the results down to exclude files.

const scanStartDir = readdirSync(join(...startDirectory))
const filteredScan = scanStartDir.filter((dir) => isDir(join(...[...startDirectory, dir])))

To keep the filter simple, we create an abstract isDir function. This function simply takes a path argument and returns a boolean for whether an individual file is a directory or not. The Node standard library has a handy statSync for getting information about files. We use it's isDirectory() function which returns true or false.

With our filtered results we can now start evaluating conditions. Our findDirectory function is written recursively which means it calls itself internally to solve subsets of the original problem.

Our first condition is the base case which is that we found the directory that we were search for so we can stop searching.

if (filteredScan.includes(searchName)) {
  return join(...[...startDirectory, searchName])
}

We also have secondary base case which is that the current directory that we are searching does not contain any directories so we can stop searching.

else if (filteredScan.length === 0) {
  return "";
}

Apart from the conditions that should stop our search, or in other terms our base cases, we need to continue searching all of the directories in our filtered search results. Here is where the recursion happens.

else {
  let subScanFound = "";
  filteredScan.every((dir) => {
    const result = findDirectory(searchName, [...startDirectory, dir]);
    if (result.length > 0) {
      subScanFound = result;
      return false;
    }
    return true;
  });
  return subScanFound;
}

The Issue of Halfway Committing To Typescript as a Team

Delightful Engineering — Mon, 01 Aug 2022 15:46:00 +0000

It's generally been said that Typescript is helpful when codebases are large and complex, or when a team of people are contributing to them. A quick way for Typescript to be detrimental to a project is when the team only partially commits to it.

This applies to both libraries and applications, but library authors should be even more cognizant of embracing good Typescript practices - after all the point of libraries is for others to use them whereas users of an application don't really care if the types and interfaces are correct. In both cases, type safety can make systems more robust and easier to use.

What does Halfway Committing To Typescript Mean?

In no specific order, here are some behaviors that would qualify as halfway committing:

Using the any type. Using any is not necessarily a bad thing for quickly prototyping a new feature or function, but for moving to production it just shouldn't be acceptable. This should be a slap on the wrist if they slip into a protected branch, or better yet should be corrected in a merge request.
Using @ts-ignore and @ts-nocheck. The one case where these are useful and acceptable are when porting over a large untyped file, but again shouldn't be merged into a protected branch or production. Major slap on the wrist if these make it into production.
Adding new Javascript files. Maybe the project is in the process of migrating from Javascript to Typescript in which case the codebase may still have some Javascript files that need to be converted. But if the team has made a commitment to Typescript, why prolong the migration by adding more Javascript?
Not using strict compiler settings. By not using strict compiler options, the project and the team is losing all of the benefits of using Typescript.

Why does Halfway Committing To Typescript Happen?

There are common reasons for why this happens:

Deadlines. Business expects that features get shipped on time and provide value. This pressure may be heavy on the team so corners may get cut to relieve that pressure.
Previous patterns are followed. This is one of the main problems with allowing any types to slip into shared branches. New developers in particular tend to pick up and follow behaviors of the team, so bad Typescript habits multiply.
Lack of proficiency. It may be the case that developers on the team need time to learn and practice Typescript.
Laziness. Perhaps members of the team just want to get the job done in the easiest way possible and don't want to spend the time and effort.

The Problems That Halfway Committing To Typescript Causes

Any tool, technology, or new library is not a silver bullet, there are always tradeoffs and this is true for Typescript.

Here are some of the tradeoffs.

Benefits:

Type safety. Typescript is a tool that helps you to enforce type safety, that is its primary job. Those hundreds, thousands, or even millions of tiny edge cases are simply not possible for a human to catch. Typescript is good at catching these and will likely help you sleep better at night.
In-editor documentation. This one is particularly helpful for consumers of libraries. Autocompletion of arguments and return types is one of the best features of Typescript and is a huge time-saver. Most developers either don't like to, or don't have time to manually update documentation in parallel, this is a great way to solve that problem. Especially for large data interfaces this is key.
Code quality. This goes hand-in-hand with Type safety but deserves it's own mention. One can absolutely write bad code in any language, but the practice of being intentional about data types makes one slow down and understand what the implementation is doing.
Clear I/O interfaces. The contract of data going in and data coming out of a system is important and always helpful to the user. This is the whole reason why things like USB ports and Mac lightning ports are useful, users of the adaptors can use the same interface to communicate with their devices. This should be the same case with software systems.
Short feedback loop. Iterative development relies on fast feedback loops. The Typescript compiler is great at this - it will throw red squiggles when it flags an issue. Better to catch these things as fast as possible as opposed to some issue going out into production and having to wait for the next release to get the fix out.

Sacrifices:

More code. There will inevitably be more code in a Typescript project when compared to one with Javascript, and especially in libraries there can be thousands of lines for types and interfaces alone. More code is more maintenance and complexity.
Longer build times. In most cases an additional compilation step is added to the build process, especially for applications that are running in a browser. Depending on the size of the codebase, build times could be much longer. There are however many tools nowadays that speed up builds, like esbuild and swc.
Learning curve. For developers coming from a Javascript background, they may be unfamiliar with static typing. For those coming from Java, Go, or other typed languages, the learning curve may be smaller. It costs time and money for developers to learn how to use Typescript.
Initial setup cost. It's not always straightforward to get Typescript set up. It's also not always easy to get Typescript to work with a particular toolchain, and some tools may require a lot of configuration. Overtime this seems to be getting easier as adoption grows with projects like ts-node, deno, bun.js where Typescript is a first-class citizen, but this one shouldn't be overlooked.

By only halfway committing, the team may be sacrificing all of the benefits
while also accepting the tradeoffs. If those few bad behaviors mentioned earlier continue to happen, the codebase will likely get much larger and more complex, build times will get worse, and even more code will be untyped and not any safer. Every time a developer introduces another any type into the codebase, you can kiss goodbye to type safety, in-editor documentation, any compiler warnings, and that sweet I/O interface. Overtime this has major consequences for the health of a project.

In other terms, this behavior is like buying an expensive private jet and never using it, but paying for all the overhead like fueling it and paying for the insurance.

It's probably better to not start using Typescript until the team is fully aligned and ready to commit to avoid the above problems. Don't invest in the private jet if you aren't gonna use it.

Quick tip:

One way to avoid this bad behavior is to write up a team contract or a set of guidelines that people can follow and include it as part of a merge request checklist. If that's not enough, you can use es-lint to enforce the rules (no-explicit-any rule)[https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/eslint-plugin/docs/rules/no-explicit-any.md].