DEV Community: Harsh Pandey

Scalable WebSocket Architecture

Harsh Pandey — Wed, 28 Sep 2022 16:12:48 +0000

Overview

The web has rapidly evolved from its origins of websites serving static HTML content, to today's highly dynamic web applications. Whether it's games, chat, or productivity tools, users expect the apps to be up-to-date and enable real-time interactions with others.

WebSockets, which enables real-time communication from within web browsers, are one of the central pieces that has enabled this level of interactivity. And yet, despite the wide spread adoption of WebSockets, surprisingly little has been written about effective ways to implement them at scale. Most current efforts try to retrofit techniques proven to scale traditional request/response applications (stateless) to real-time/streaming use cases (stateful), leading to architectural complexity and limitations.

In this post, we will explore potential architectures for a typical realtime application: a turn-based multiplayer game. We'll start first with the commonly recommended architecture from the massively popular WebSocket library Socket.IO, and then we'll propose an alternative architecture and detail some of its benefits.

Architecture Options

Setup

We are designing a backend system for a real-time turn-based game similar to online Chess or Monopoly. Users are grouped into sessions or "rooms" together, where users within a room interact with each other and there can be many concurrent rooms at any given time (in the chess example, a live chess match corresponds to an active room). To support a large number of concurrent rooms, we'd like a horizontally scalable architecture where adding additional game servers increases the number of concurrent rooms/players that our system can support.

Traditional Socket.IO architecture

The most commonly cited architecture for scalable WebSocket backends can be found on the Socket.IO documentation. Players are routed via a generic load balancer to one of many available server instances, and the server instances communicate with each other via a message broker. State is maintained in a globally shared database.

Data Flow:

A user from US East sends a message (e.g. a chess move) over a WebSocket connection scoped to a particular room/session (e.g. wss://example.com/room1). The connection is established with a load balancer like nginx, HAProxy, Amazon CloudFront, etc.
The load balancer forwards the messages to a custom WebSocket server. When initially establishing the connection, the load balancer has to choose which WebSocket server instance to forward the connection to. To reduce latency, it should ideally pick one in the closest region to the user (using something like GeoDNS).
The WebSocket server processes the message and issues a write query to a globally available database system like Postgres in order to update the game state. The database operation must be atomic to account for the fact that writes may also be coming from other WebSocket servers.
The WebSocket server sends the game state update (e.g. the current chessboard position) to a message broker like Redis or RabbitMQ so that users connected to other WebSocket servers can receive the update.
The message broker broadcasts the state update to all connected WebSocket servers.
The WebSocket servers forward the state update (via the load balancer) to all WebSocket connections for the relevant room.
The load balancer forwards the update over the client WebSocket connection.

Architecture using a Stateful Router

This is an alternative architecture which relies on a specialized stateful proxy to group users to server instances based on their room. This "colocation" of users localizes data to the server instance and eliminates the need for a separate message broker.

Data Flow:

A user from US East sends a message over a WebSocket connection scoped to a particular room/session (e.g. wss://example.com/room1). The connection is established with the Stateful Router.
The proxy forwards the messages to a custom WebSocket server. When initially establishing the connection, the proxy selects a WebSocket server for the requested room based on proximity to the user. For all subsequent connections to this roomId, the Stateful Router uses its internal map to route the connection to the same WebSocket server.
The WebSocket server processes the message and issues a write query to a database in order to update the game state. Note that because all messages in the room are processed by a singular server, this database can be local to this server – this can be as simple as updating some in-memory data structure inside the server process, or a locally embedded database like SQLite/RocksDB.
The WebSocket server sends the game state update to the proxy.
The Stateful Router broadcasts the state update to either one or optionally all connected WebSocket clients in the room.

Advantages of Stateful Router

Migrating from a message-broker based architecture to one utilizing a Stateful Router offers a number of advantages:

1) Fewer messages (i.e. improved scalability)

Before: Each WebSocket server needs to process every state update, filter messages addressed to connected clients and forward it along. Thus the scalability of the system is limited by the maximum throughput of a single server for processing state updates.

After: Only one server needs to process the messages and state updates for a given room. As long as the largest/busiest room can be handled on a single server, this architecture is infinitely scalable.

2) Reduced server load (via connection multiplexing)

Before: Load balancers and servers maintain dedicated WebSocket connections for each client connected to them. This consumes resources (CPU, memory, file descriptors) on the servers.

After: Servers only have a single connection to the Stateful Router. The proxy combines all incoming and outgoing messages over this single connection – this technique is known as connection multiplexing and it serves to shield the servers from concurrent user load. No more worries about file description limit and memory constraints.

3) Data locality (via client colocation)

Before: Servers write to a shared database and need to ensure a data consistency model (e.g. via atomic write operations or CRDTs).

After: The data for a room is local to the server, allowing use of local in-memory or embedded databases – consistency becomes trivial. Additionally, if a persistence layer is used, it would only need consistency guarantees within a geography, without having to worry about global replication lag.

Hathora

At Hathora, our mission is to make it easier for developers to build, launch, and scale multiplayer games. One of the core technologies we have built is the Hathora Coordinator, which is our fully managed multi-tenant implementation of a Stateful Router.

Hathora Coordinator

An overview of the Hathora Coordinator and how it's used to power multiplayer games can be found in the Hathora documentation. In addition to the stateful routing functionality described in this article, the Hathora Coordinator also implements a few other key features:

Edge networking

To minimize latency, it's important to accept and process traffic closest to where users are (we wrote a blog post going into more detail about this). The Hathora Coordinator is deployed across 8+ regions globally and accepts traffic at the edge before routing it to the right server instance.

Transports

We talked about WebSockets in this article, but the same techniques apply to other persistent transport types as well. The Hathora Coordinator also supports direct TCP connections, with UDP and WebTransport support coming soon. These connections are handled entirely by the Coordinator in a way such that your servers don't know or care about which transport type the clients originally connected with, greatly simplifying your servers networking code.

Analytics

Since all traffic goes through the coordinator, it has access to important information like the number of concurrent connections over time, which geographies the connections originate from, the average connection duration, etc. Having access to this information can be invaluable for understanding your user base and engagement.

Hathora: Multiplayer Made Easy

Harsh Pandey — Tue, 19 Apr 2022 16:21:16 +0000

More multiplayer games are being developed than ever before, but building & launching a successful online multiplayer game remains one of the most notoriously difficult endeavors in the software world. The biggest challenges developers face are (1) choosing and correctly implementing the technologies to enable multiplayer, and (2) ending up with an architecture that you can operate and scale.

If you are starting a new game from scratch, you are inundated by a multitude of choices:

How should I set up my project structure?
Should I use peer-to-peer or client-server architecture?
TCP, UDP, WebSockets, or WebRTC for networking?
How should I implement authentication?
How do I persist data?

For the amount of demand for multiplayer games, the development process should not be as hard as it is today. Armed with a passion for multiplayer games and extensive experience building & operating scalable systems, we have been working on a solution. Enter Hathora: a framework for building multiplayer games.

Hathora can be used for:

🎲 Turn-based multiplayer games - Examples: Chess, Words with Friends, Codenames, etc
🎮 Realtime multiplayer games - Examples: Among Us, Agar.io, Slither.io, etc
💬 Realtime & social applications - Examples: chat apps, delivery tracking apps, etc

We've built Hathora to provide the most streamlined development experience when starting a multiplayer game from scratch. The framework provides the right abstractions such that a game can be prototyped in minutes and scaled to millions of users.

Prototype in Minutes

Hathora's philosophy is to minimize the burden on the developer by allowing them to focus only on functionality specific to their game, while the framework serves as the engine to power the remaining machinery. We are pushing the limits of game development to drastically reduce the time it takes to ship a game.

Autogenerate type-safe data model and clients

All Hathora projects start by defining their API using Hathora's language agnostic declarative API format. From this definition, Hathora generates typesafe data models and clients for the language of your choice.

Hathora frees you from the extensive boilerplate normally required for client/server applications.

Write server logic in 20-100x fewer lines of code

Write backend logic seamlessly in the language of your choice. Just read and update in memory state, no database queries or ORMs needed – Hathora automatically manages persistence behind the scenes.

Iterate faster with the Prototype UI

Test your backend logic without writing any frontend code using the built in Prototype UI. Create & join rooms, call server methods, and interact with live data all from within your browser.

Hathora enables play testing from the first minute.

Flexible, cross-platform client support

Once the backend functionality has been verified, it’s easy to use the generated client to build your UI in your platform and technologies of choice and share the game with the world.

The Hathora client fully handles networking, and easily integrates into any frontend technology/platform of your choice.

Other features you get out of the box with Hathora:

Automatic state synchronization
Optimized networking (binary protocol, delta encoding)
Built in authentication
Project structure generation & development server with hot reloading

Scale to Millions

Once you've built & released your game, you don't want to spend the next 6 months battling scaling issues and reworking the architecture to meet your user demand. Hathora includes several key components that make it so you can be production-ready from day 1.

Hathora Coordinator

Central to the Hathora architecture is the managed Coordinator service, which sits between Hathora clients and Hathora backends.

The coordinator plays several key roles in the architecture:

It ensures consistent routing so that clients playing the same game get routed to the same backend instance.
It load balances games between backend instances to ensure an even distribution of traffic.
It reduces latency by dynamically assigning games to instances based on the geographic distribution of the connecting users.
It handles backend failover by automatically migrating the games from the failed instance to a new instance, all of which is transparent to clients.

Hathora Protocol

The Hathora Protocol governs the communication between clients and servers. It is flexible, lightweight, platform & transport agnostic.

At the core of the protocol lies a highly optimized serialization format. All data is binary encoded and servers send only delta updates to clients after the initial snapshot. The encoding/decoding code is generated from the API definition and is incredibly efficient so as to minimize latency while sending and receiving messages.

Hathora Cloud

Although it will always remain possible to self host your backend instances, for those who do not wish to manage their own infrastructure we have built Hathora Cloud.

Hathora Cloud is a platform for running Hathora applications at scale. It has features like autoscaling, multi-region deployments, monitoring & analytics, and migration management. Hathora Cloud makes it simple to deploy your game servers globally and scale to millions of users.

Hathora Cloud is currently in private alpha, sign up using this form.

Get started with Hathora today.