DEV Community: Luke Westby

RedBoard: a collaborative whiteboard driven by Redis

Luke Westby — Sun, 28 Aug 2022 18:29:00 +0000

Overview of My Submission

RedBoard is a demonstration of a collaborative whiteboard app driven by Redis's real-time features.

I've been fascinated by Figma's multiplayer editing technology since 2019 when they published this article about how it works. I wanted to use this hackathon as an opportunity to attempt to implement a minimal reproduction of that technology using as much as I could infer from that article and some other material that Figma has shared publicly. I am grateful to their engineering team for taking the time to share what they have built and learned with all of us!

The high-level design of RedBoard is based heavily on this architecture diagram shared by the Figma team to demonstrate using FigJam to make such diagrams, but is of course greatly simplified and uses only Redis for storage and messaging. Journaling is implemented using Redis Streams, and checkpointing folds those streams into RedisJSON values. Not depicted in that diagram is presence, or the real-time broadcasting of actions taken by other users on the same board. Presence is implemented using Pub/Sub.

The backend is implemented using Redis and Rust, and the frontend is implemented with React and Recoil, a.k.a. the RRRR stack 🙂. The two communicate using a two-phase protocol over a WebSocket connection, and the backend app essentially constitutes a very simple real-time document store using Figma's multiplayer change coordination strategy.

This project was a great learning experience and if nothing else it helped to me truly recognize just how difficult it is to build a system like this with production quality and at large scale. Furthermore, using Redis to handle all of the data needs provided inspiration for some new ways I plan to use Redis in my professional work at StructionSite.

Submission Category:

Wacky Wildcards

Language Used

Rust

Link to Code

lukewestby / redboard

Submission for the 2022 Redis/DEV hackathon

RedBoard

A collaborative whiteboard driven by Redis, inspired by Figma. Please see my submission post for an explanation of my goals and inspiration for this project.

How it works

The backend application is essentially a very simple, generic real-time JSON store. It allows multiple clients to modify a group JSON by ID such that the last write received by the server always wins, and all clients are incrementally notified of the correct state of the objects. The visual functionality of the end-user application is built on top of this store, but the store knows nothing about the meaning of the objects that it stores.

Some important terms:

Board: a collection of viewable objects that many people can edit at once
Object: an individual visual item in a board
Change: a structured representation of an edit for an object
- { type: "Insert", id: "<UUID>", "object": { "property1": "hello", ... } }
- …

View on GitHub

Additional Resources / Info

Blog post about Figma's multiplayer technology: https://www.figma.com/blog/how-figmas-multiplayer-technology-works/
Figma multiplayer architecture diagram: https://www.figma.com/community/file/989634471195357925
Recoil: https://recoiljs.org/

Check out Redis OM, client libraries for working with Redis as a multi-model database.
Use RedisInsight to visualize your data in Redis.
Sign up for a free Redis database.

How much data do I need to have Big Data?

Luke Westby — Thu, 01 Aug 2019 04:23:02 +0000

Do you have a morning routine?

Luke Westby — Sat, 11 Aug 2018 21:53:52 +0000

I recently started the following morning routine:

Clean my room and make my bed
Do a couple body-weight exercises
Stretch
Drink a glass of water and take vitamins
Shower and get dressed

I do it the same way every single day. The difference it has made in my outlook and ability to focus for the rest of the day is noticeable. Do any of you do something similar with your mornings?

What’s something that should be obvious that developers seem to misunderstand?

Luke Westby — Sun, 08 Jul 2018 20:08:07 +0000

I found this on Twitter, but I don’t have a twitter account so I thought I’d bring it over here. It seems like a good fit for #discuss!

// Detect dark theme var iframe = document.getElementById('tweet-1015086438277435392-184'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1015086438277435392&theme=dark" }

How do you stay focused when everything sucks?

Luke Westby — Wed, 27 Jun 2018 22:42:44 +0000

When the news is really bad it can feel nearly impossible to care about or do anything else. I would love some thoughts from this community on how you keep up with life in the face of all the bad stuff.

Discuss: Why do developers equate popularity with success for OSS projects?

Luke Westby — Mon, 25 Jun 2018 02:09:20 +0000

Heads up! This post has some claims that lack citations. The power dynamic here is such that I'd rather be dismissed to generally no effect than to put the heat on some developers without a platform who write internet posts that I think are wrong.

Anyway, let me frame up this discussion a bit:

I am a member of the Elm core team. I made a significant outlay of personal effort to reach this level of contribution, so I think it’s reasonable to claim that I understand what it’s like to care about a project and desire for it to succeed.

A lot of the online lobbying directed at us for changes in prioritization is based on the idea that Elm is not successful unless its community is growing faster now than it was some amount of time ago. This commentary rarely explains how the poster personally defines “success” for Elm, so I have to assume that adoption and success are one and the same in their mind.

For Elm, we reject the idea that adoption is success or even a very good reflection of success. Elm seeks to be an exceptionally good programming language and platform for building front-end web applications. It is successful to the degree that it does that. Major, concrete steps toward this goal will not always result in community growth, especially under Elm's long release cycle. That is okay!

As a final thought, here is an illustrative example of my confusion: if we grant that adoption and success are related, then what does it mean to be the most popular and thus most successful project? If Elm were the most-used front-end technology, what prize would the Elm community win?

So. What do y'all think is the cause of this apparent linkage in the mind of The Developer?

I think that if I can better understand why people feel this way then I will be better at explaining how to measure Elm's success to the Elm community.

If you would like to share your thoughts anonymously you can email me at hello@lukewestby.com.

Ellie Weekly Update 2017-10-28

Luke Westby — Sat, 28 Oct 2017 15:26:53 +0000

Hi friends! Not a ton of stuff happening right now since I've been traveling and attending to personal things. Here's this week's brief update:

New Features

Added a section in the bottom right to view calls to Debug.log.

Community

Deleted Ellie's Twitter account. If enough people are interested I can write about why I chose to do this.

In Progress and Experiments

Started rewriting the build manager in PureScript.
Began reworking the way Ellie determines if a revision is yours in order to prepare for GitHub auth, Gist import, and a whole bunch of other stuff.

What do you build to learn a new language or style?

Luke Westby — Mon, 16 Oct 2017 15:01:59 +0000

I like to reimplement Elm's Json Decoders in whatever language or framework I'm trying to learn. I did this when I was getting up to speed on fantasyland in JS, and when I needed to brush up on Python, and now this week with PureScript. How about you?

Ellie Weekly Update 2017-10-14

Luke Westby — Sat, 14 Oct 2017 15:39:12 +0000

Hi friends! I wanted to keep you all updated on what's happening with Ellie. I'll publish a roundup each week that I actually make changes and then archive the related cards in Trello. Here's this week's update:

New Features

Put some links in the header. You can find links to Twitter, GitHub, and Trello at the top right of the app.
The open section in the sidebar can be closed.
Replaced the message in the Ready screen with a button to start compiling.

Community

Added a README in the repo that contains info about how to develop Ellie locally.
Created a contributor license agreement and linked it to the repo with CLA Assistant
Got the live stream going again, check out Friday's stream here

Finance

Applied to Open Collective

In Progress and Experiments

Adding a section to view calls to Debug.log
Learning PureScript to see if it's worthwhile to use it to rewrite the build manager.
Considering options to abstract the S3 dependency so people don't have to set up a bucket in order to do local development.

Why Ellie Will Not Support Multiple Files

Luke Westby — Wed, 11 Oct 2017 16:07:29 +0000

Ellie, the Elm Live Editor, lets you use the Elm platform in your browser. I released it in February of 2017 as a way to make it easier for the Elm community to share code. Despite it's popularity within the still small and growing Elm community it remains a side project so I need to be judicious about how I spend my time working on it. Its popularity also portends its importance to the community and I believe that comes with a responsibility to develop it in a way that meets the needs of the community.

Ellie isn't just a tool for sharing Elm code; it's becoming a centerpiece for online, interactive discussion of Elm. I get a push notification whenever someone shares an Ellie link in the Elm Slack. A typical weekday includes at least one conversation where participants share Ellie links back-and-forth to help someone arrive at a better understanding of an Elm concept. At Oslo Elm Day this year I claimed that in order to write good software we need to understand how people use it. Several months later at Elm Conf US I shared that conclusion about Ellie specifically: Ellie enables conversations about Elm.

This discovered identity guides Ellie's mission and will inform all of my decisions about features and UX going forward. This led to the planned addition of real-time collaboration. It may also lead to the addition of more ways to personalize the Ellie UI, including the addition of more familiar editor keymaps like Emacs and a light color theme. Beyond positive additions, though, it also informs the decision not to include certain features even in the face of user requests.

Of course I'm talking about including multiple files in an Ellie project. This is among the most requested features. But, like Elm itself, the development of Ellie is not driven by direct democracy. I don't believe that allowing the inclusion of multiple files meets the criterion of enabling better conversations about Elm online.

Here's why: if you ask me a question about a specific concept under a broad topic, you probably won't appreciate it if I send you a link to a 4000 word Medium post. That would end the conversation! I want to respond with brevity so that you can consider what I said and build on it so we can keep moving. A chat application is a much better analogy for understanding Ellie's primary use case. In both cases you can type as much as you want, but the medium encourages you to be brief because you are in a dialogue.

I've heard a counterpoint that some concepts can't even be explained briefly in Ellie because they depend on having multiple files. Examples include the Elm module system itself and opaque types. My response to this is is just that the number of concepts in this category is pretty small and I'd rather Ellie be unusable in this small set of cases than split the focus of Ellie between concise conversation and large projects, making it worse at both.

Perhaps someone else will build software that allows people to build and run large Elm examples online. It may even one day be a project related to Ellie, just as CodePen introduced CodePen Projects. However, the core focus of Ellie is conversation and optimizing for that is more important to me than making it a catchall for every code-sharing need.