Deloy SocketIO Server Using Docker and Nginx Load Balancer (+SSL)

In my last project, I needed to have socket servers that could manage connections for more than 10 thousands of mobile applications. I also needed to make the connection using a secure connection if possible. After digging into some scenarios, I finally can manage the socket server deployment using Docker and Nginx in a single Virtual Private Server.

Here are the steps that I did:

Project Structure

This is how I organize the socket server project

/socket-server
|-- src/
    |-- index.js
    |-- loggers/
|-- .env
|-- .env_one
|-- .env_two
|-- .env_three
|-- .env_four
|-- Dockerfile
|-- docker-compose.yaml
|-- nginx.conf
|-- package.json
|-- yarn.lock

The index.js

import express from "express";
import { createServer } from "http";
import { Redis } from "ioredis";
import { Server } from "socket.io";
import { createAdapter } from "@socket.io/redis-adapter";
import logger from "./logger/winston_logger.js";
import bodyParser from "body-parser";

const app = express();
const http = createServer(app);

const port = process.env.PORT;
const serverName = process.env.SERVER_NAME;
const redisHost = process.env.REDIS_HOST;
const redisPort = process.env.REDIS_PORT;
const socketKey = process.env.SECRET_KEY;

let numConnectedSockets = 0;

const pubClient = new Redis({
  host: redisHost,
  port: redisPort,
  keyPrefix: "myapp",
});
const subClient = pubClient.duplicate();

logger.info(
  `Redis client connected to ${redisHost}:${redisPort} with status: pub: ${pubClient.status} sub: ${subClient.status}`
);

// listen to redis connection status
pubClient.on("connect", () => {
  logger.info("Redis pub client connected!");
});

subClient.on("connect", () => {
  logger.info("Redis sub client connected!");
});

// create socket io server with adapter
// add your other services here
const io = new Server(http, {
  cors: {
    origin: ["http://localhost:3000", "http://localhost:8020"],
    methods: ["GET", "POST"],
  },
  adapter: createAdapter(pubClient, subClient),
});

// express listen to port
http.listen(port, () => {
  logger.info(`Server ${serverName} is running on port ${port}`);
});

// socket io connection
io.on("connection", (socket) => {
  logger.info(`User connected: ${socket.id} to socket server ${serverName}`);

  socket.emit("hi", {
    serverName: serverName,
    msg: `Hello from socket server ${serverName}!`,
  });

  // socket io events
  socket.on("disconnect", () => {
    numConnectedSockets--;
    logger.info(`User disconnected: ${socket.id} from ${serverName}!`);
  });

  socket.on("hello", (msg) => {
    logger.info(`Receive a hello from ${socket.id} with ${msg}`);
    logger.info(`server socket key is ${socketKey}`);
    logger.info(`test socket key vs msg is ${socketKey} vs ${msg}`);
    if (msg) {
      if (msg === socketKey) {
        logger.info(`User ${socket.id} sent the correct key. Welcome!`);
        socket.emit("introduce_yourself", {
          serverName: serverName,
          msg: `Please introduce yourself!`,
        });
      } else {
        socket.disconnect();
      }
    } else {
      socket.disconnect();
    }
  });

  socket.on("introduction", (msg) => {
    logger.info(`Receive a introduction from ${socket.id}`);
    if (msg) {
      const { name, token } = msg;
      if (!name || !token) {
        logger.error(
          `User ${socket.id} did not introduce themselves correctly!. Disconnecting...`
        );

        // disconnect user
        socket.disconnect();
        return;
      }

      logger.info(
        `User ${socket.id} introduced as ${name} with token ${token}`
      );

      // validate name and token
      let isAuthorized = false;
      if (name === "admin" && token === "admin") {
        isAuthorized = true;
        logger.info(`User ${socket.id} is an admin!`);
      }

      if (token === socketKey) {
        logger.info(`User ${socket.id} is an authenticated user!`);
        isAuthorized = true;
      }

      if (isAuthorized === false) {
        logger.error(
          `User ${name} with socket id ${socket.id} is not authorized. Disconnecting...`
        );

        // disconnect user
        socket.disconnect();
        return;
      }

      numConnectedSockets++;
    } else {
      logger.error(
        `Can not get the message from ${socket.id} while introduction. Disconnecting...`
      );

      logger.error({
        msg,
      });

      // disconnect user
      socket.disconnect();
    }
  });

  socket.on("log_event", (data) => {
    socket.broadcast.emit("broadcast_log_event", {
      serverName: serverName,
      msg: data,
    });
  });
});

// error handling
io.on("error", (error) => {
  logger.error(`Error: ${error}`);
});

app.use(bodyParser.json());

// express default route
app.get("/heartbeat", (req, res) => {
  res.sendStatus(200);
});

// base url
app.get("/", (_, res) => {
  winstonLogger.info(`base endpoint called`);
  res.status(200).json({
    error: false,
    msg: "Base endpoint works",
  });
});

app.get("/", (req, res) => {
  res.status(200).json({
    message: `Welcome to socket server ${serverName}`,
    socketConnected: numConnectedSockets,
  });
});

As you may noticed here, we are using the Redis adapter to keep the socket servers synced when we have incoming and upcoming events emitted. You can go to the official documentation here for the Redis adapter.

Nginx Configuration

The nginx configuration goes like this

# Reference: https://www.nginx.com/resources/wiki/start/topics/examples/full/

worker_processes 4;

events {
  worker_connections 1024;
}

http {
  server {
    listen 80;

    location / {
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header Host $host;

      proxy_pass http://nodes;

      # enable WebSockets
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "upgrade";
    }
  }

  upstream nodes {
    # enable sticky session with either "hash" (uses the complete IP address)
    hash $remote_addr consistent;
    # or "ip_hash" (uses the first three octets of the client IPv4 address, or the entire IPv6 address)
    # ip_hash;
    # or "sticky" (needs commercial subscription)
    # sticky cookie srv_id expires=1h domain=.example.com path=/;

    server server-one:3000;
    server server-two:3000;
    server server-three:3000;
    server server-four:3000;
  }
}

Dockerfile

# Use the official Node.js 21 image as the base image
FROM node:21-alpine

# Set the working directory inside the container
WORKDIR /app

# Copy package.json and package-lock.json to the working directory
COPY package*.json ./

# Install the app dependencies
RUN yarn install --production=true

# Copy the rest of the app source code to the working directory
COPY . .

# Set the non-root user to run the application
USER node

# Expose the port on which the app will run
EXPOSE 3000

# Start the application
CMD node --env-file=.env src/index.js

Docker Compose

services:
  nginx:
    image: nginx:alpine
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    links:
      - server-one
      - server-two
      - server-three
      - server-four
    ports:
      - "3002:80"

  server-one:
    build: .
    expose:
      - "3000"
    env_file:
      - .env_one
    restart: always

  server-two:
    build: .
    expose:
      - "3000"
    env_file:
      - .env_two
    restart: always

  server-three:
    build: .
    expose:
      - "3000"
    env_file:
      - .env_three
    restart: always

  server-four:
    build: .
    expose:
      - "3000"
    env_file:
      - .env_four
    restart: always

Nice, then we can start deploying the server using the command docker compose build and following with the command docker compose up -d .

Host Machine Nginx + Letsencrypt

To be able to use the wss://mydomain for the socket URL when the socket client wants to connect to the server, we can set up the SSL using Letsencrypt in the host machine. I will write about this in a separate post.

After completing the SSL setup, then we can add this block of configuration in the nginx configuration inside the server { block. Usually at /etc/nginx/sites-available/default file.

location /socket.io/ {
   proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
   proxy_set_header Host $host;

   # the port of the nginx in docker compose
   proxy_pass http://localhost:3002;

   # enable WebSockets
   proxy_http_version 1.1;
   proxy_set_header Upgrade $http_upgrade;
   proxy_set_header Connection "upgrade";
}

That’s it, then your socket client will able to connect using the wss://mydomain socket server URL.

If you are using Flutter. There is a socket io client plugin that you can use. Check it out here. Below is the code to connect to the socket server using SSL enabled.

import 'package:socket_io_client/socket_io_client.dart' as io;

// create a new socket instance
socket = io.io(socketServer, {
  'transports': ['websocket'],
  'secure': true,
});

ocket.onConnect((_) {
  debugPrint('get connected to the server $socketServer');
});

socket.onError((error) {
  debugPrint('error: $error');
});

socket.onDisconnect((_) => debugPrint('disconnected from $socketServer'));

I hope you found this post useful. Happy coding!