Synchronizing Redux logs with server via WebSocket using Logux

Alexey Ermolaev — Wed, 11 Dec 2019 23:25:02 +0000

This is rather lengthy and personal story of why a small dev team decided to use Logux in production and what problems did it solve. It's more about fullstack application architecture and not code heavy guide or How To. Logux docs do this part better than I could ever do. TL;DR: If you ever struggled with sockets, Redux and keeping your data real-time and synced across clients and tabs, you should definitely try Logux.

Disclamer:
Our project isn't even released yet, so we dare to experiment and rewrite some parts of our codebase as we want. This story should not be viewed as a guide on how to migrate your huge enterprise codebase to Logux, rather a story of why you should give it a chance in your next project.

Part 1. The Problem

Imagine you are a small dev team of five, who just recieved project from outsource team. And in that project there are chats.

In first version, that our outsourcers left for us to develop, chats stack consisted of this:

SQL database
PHP backend
Redis database
NodeJS backend
React+Redux frontend

How the hell did this thing work?

Something like this:

User opens chat
Frontend sends GET request to PHP and simultaneously connects via socket to NodeJS, passing chat id.
PHP gathers all messages from SQL and sends them to frontend
In parallel NodeJS subscribes to chat/:id channel in Redis
User sends new message
Frontend sends POST request to PHP
PHP adds message to SQL (for long term storage) and to the channel in Redis
NodeJS gets message via subscription and send it through socket to frontend

This is so wrong on so many levels. First of all why would you choose Redis as means of communication between NodeJS and PHP? It's fast while it's on a single machine, but this story doesn't scale good. It almost doesn't scale at all.

So I decided to ditch PHP and Redis altogether and transfer all backend chats logic in NodeJS. PHP remained in project since it contains main part of application logic, and Redis is used as session and cache storage. But none of them now have part in chats.

On frontend I wanted to keep socket connection to a minimum, so I added SharedWorker which connected to a socket and proxied all messages between tabs and NodeJS. But this one change almost tripled amount of bootstrap code since SharedWorkers is not widely supported by browsers and I had to keep socket logic in "main thread" while doubling it in SharedWorker.

In second version of chats we got something like this:

Things I didn't like in this version were:

We still had several socket connections if browser didn't support SharedWorker
We still had separate connections to PHP and NodeJS
Now chats logic is separated from application logic and there are two backends connected to SQL

So it wasn't perfect but it worked and was better than that Redis thingy.

And then, a month or two later, we realise that we need real-time communication on several pages of our project besides chats, which meant that we should add tremendous amount of logic to sockets/workers and somehow we should send data from PHP backend through NodeJS server in order to achieve this real-time communication without rewriting all our application in NodeJS.

Somewhere near this time I was rewatching Andrey Sitnik's talk about Logux and realisation hit me. We could use Logux for solving almost all our issues with chats, real-time communications and cross-tab sync!

Part 2. The Solution

Logux is a set of libraries and NodeJS server framework designed by Andrey Sitnik (author of awesome PostCSS and Autoprefixer) which proposes somewhat new method to communicate between client and server in real time.

Imagine if your server had Redux-like logs and reducers and you could synchronise that logs across clients and between tabs using sockets and localStorage. Logux does exactly that and with logux/redux package it connects to Redux in no time. Logux also provides a "channels" functionality so we could split all our actions in neat channels, like one channel per chat. And on top of that it has offline functionality and automatically synchronises when connection is restored.

Also Logux has awesome docs site: https://logux.io. Check it out!

So what perspectives opens up ahead of us with this knowledge?

From frontend dev perspective our app now looks like pure Redux app. Every piece of data from server now comes in form of Redux action. Every action that needs to be performed on server now is dispatched as Redux action and synced via Logux.

This approach grants us ability to optimistically update UI, even if action is not yet processed on server. Even if we are offline. And in case of error Logux will simply "undo" that action and remove it from Redux log.

From backend perspective, however, thing doesn't look great right now. Looks like we have to rewrite all our backend to NodeJS-based Logux server, right?
Wrong! Logux got it all covered: it has http communication protocol. You simply specify endpoint and now you can process all your actions on any backend you like as long as it can communicate via http. Also you can add new actions from your backend to Logux via http request.

And voila, our complex structure boiled down to this simple chain:

Tabs synchronises through the leader tab which is elected on start and reelected if previous leader tab is closed or not responding. So we always have one connection per client. All application logic finally migrated back to PHP and Logux serves as sockets proxy for PHP.

Adding new real-time features to our frontend part became simpler than it could be, while PHP got means to communicate to client in real-time without major refactoring.

Part 3. Enough talking, show me the code!

First of all, let me show you a small glimpse of how chats logic on client looked before Logux:

const init = () => {
  if (typeof SharedWorker !== 'undefined') {
    this.worker = new SharedWorker('/chat-worker.ts')
    this.worker.port.start()
    this.worker.port.onmessage = processWorkerMessage
    this.worker.port.postMessage({
      type: 'connect',
      data: user.id
    })
  } else {
    this.socket = openSocket(domain, { path: '/ws/userChats', query: `user_id=${user.id}` })
    this.socket.on('connect', connected)
    this.socket.on('connect_error', disconnected)
    this.socket.on('connect_timeout', disconnected)
    this.socket.on('user-chats', setUserChats)
    this.socket.on('chat-messages', fillChatMessages)
    this.socket.on('message', newMessage)
    this.socket.on('logout', logout)
  }
}

protected processWorkerMessage = (event: MessageEvent) => {
  const { data: { type, data } } = event
  const actions = {
    'connected': connected,
    'disconnected': disconnected,
    'user-chats': setUserChats,
    'chat-messages': chatMessages,
    'message': newMessage,
    'logout': logout
  }

  if (actions.hasOwnProperty(type)) {
    actions[type](data)
  }
}

As you can guess all called functions just dispatched actions in Redux. And in SharedWorker there were same amount of socket.on() code. Maybe it could be written better but, let's be honest, do you have time to perfect your code in strict deadlines that you already failed? And, should I remind you that amount of message types were about to grow exponentially because of new functionality that we were about to add.

So, now, when you saw the part of horror, here's tiniest guide to logux-redux:

import createLoguxCreator from '@logux/redux/create-logux-creator'

const createStore = createLoguxCreator({
  subprotocol: '1.0.0',
  server: process.env.LOGUX_SERVER || 'ws://localhost:31337',
  // Probably Andrey thought that setting default Logux port to "ELEET" would be funny
  userId: localStorage.getItem('user_id') || false,
  credentials: localStorage.getItem('logux_token')
})

const store = createStore(reducer, preloadedState)

createLoguxCreator returns familiar Redux function createStore which in turn return store binded with Logux that you can use any way you like.

DONE! You're awesome!

If you did it right your frontend is now connected to Logux server and you can subscribe to channels using familiar dispatch with a little twist:

dispatch.sync({
  type: 'logux/subscribe', // Use 'logux/unsubscribe' to unsubscribe
  channel: channelName // i.e `chat/${id}`
})

Notice that dispatch now has sync function to dispatch action that should be synced to server and between tabs and crossTab function if you want to synchronise between tabs without sending action to server. If you don't need any synchronization for particular action you can use dispatch.local.

I'll restrain myself from showing you the server part. Logux documentation explains it all perfectly. Seriously, go and check it out now if you didn't already!

The one thing that I will mention about backend: we had to write our own PHP library for communicating to Logux, and adapter for Laravel. They're still in early stages but we're using them in production. (please forgive our backend dev, he can't write commit messages)

Part 4. The Downsides

Lacking support for TypeScript (no worries though, we've entered TypeScript era and I'm sure support will arrive some day in the near future)
You cannot reconnect to Logux with different credentials without reloading whole page. This, in turn, messes all authorisation architecture of your SPA. I've created issue on this and Andrey reported that he will see into it
If you are using React.Suspense for loading your initial page data you can't rely on Logux here, since you can't properly await for WebSocket message. In our project we decided to keep initial fetch requests to PHP for every loaded page, but it's relatively small trade off for convinience of Suspense.

Part 5. The Conclusion

Logux is pretty cool way for a Redux application to communicate with server. It saves time and code, provides optimistic UI, offline mode and cross tab sync.
The sole idea of syncing logs may affect the way we look at client-server communications in future.
If you're writing Redux apps, you should definitely try it out for yourself on some pet project and maybe even incorporate it in the architecture of your next big project. At current state I see Logux as production-ready solution, taking into account all aforementioned downsides (most of which will definitely be fixed in next versions).