DEV Community: Omer Davidson

Let's Design Google Docs

Omer Davidson — Fri, 14 Aug 2026 15:00:00 +0000

Building a Google Docs clone is a great system design exercise. The first version is easy, and then every step after it breaks something we already built.

The requirement sounds simple. Multiple users editing the same document. Let's see how it goes.

The First Version

Our server needs to push. Someone else's keystroke has to show up on our screen without us asking for it, so we are using websockets. That part is obvious and we are not going to spend time on it.

What Happens When We Add a Second Server?

It works fine, until we get some real traffic.
What happens when a single server can't handle the traffic? What if there is an error and the only server we have is down?

To handle that we deploy multiple servers behind a load balancer.

Here is our problem. Alice connects and lands on server A. Bob connects and lands on server B. They are both in the same document. Alice types something. Server A looks up who else is in that document, finds only Alice, and forwards her edit to nobody.

Each server only knows about its own connections, and a document's users are now spread across all of them.

Redis Pub/Sub

We need our servers to tell each other about edits.

We could have every server hold a connection to every other server, but then each instance needs to know the current list of instances, and that list changes every time we deploy or autoscale. That is a service discovery problem we don't want to own.

So instead we put a message bus in the middle, and the usual choice here is Redis Pub/Sub.

What Is Redis Pub/Sub?

Redis Pub/Sub is a live broadcast.

A client subscribes to a channel by name. From that moment, anything anyone publishes to that channel gets pushed down that subscriber's connection. That is the whole feature.

We let Redis do all the filtering and the server only gets updates that it actually cares about.

The important part is what happens when nobody is listening. If we publish to a channel with no subscribers, the message is simply gone. Redis does not save it, does not retry it, and does not remember it ever existed. When we publish, all we get back is the number of subscribers that received the message.

This is called at most once delivery. It is what makes pub/sub fast and simple, since Redis is not writing anything to disk or tracking who acknowledged what. It also means pub/sub can never be our source of truth. It is a way to move messages between our servers, not a place anything is actually kept.

Using It for Our Documents

Now we can fix our two server problem. Each document gets it's own Redis pub/sub channel. The server subscribes to the documents it is using. And when an edit comes in it publishes to that channel instead of writing to the local sockets.

We subscribe when the first local client opens a document and unsubscribe when the last one leaves.

One thing worth noting. When a server gets an update, it should not deliver it to its own sockets, it should just publish it. Redis will send it right back to the same server (and other subscribed servers) and only then can we start handling it like we handle every update. This saves us from maintaining a handler for local updates and a different handler for external updates.

Now edits reach everyone, no matter which server they are connected to, and our servers still don't know anything about each other.

Two People Typing at the Same Time

We can move messages between any two users now. Unfortunately the messages are wrong.

Alice and Bob can edit the same text at the same time. Just like in Git, we get a conflict. In Git we give the second editor the choice of how to handle it. Of course we can't do that in a text editor.

Use a CRDT (Conflict-free Replicated Data Type) and let a library like Yjs or Automerge deal with it.

What is worth knowing is how that choice changes our architecture. A CRDT gives every character we insert an id that never changes, so an operation becomes "insert this character after that character" instead of "insert at position 5". Those operations commute, which means clients can apply them in any order and still end up with the same document.

That is a big deal for us. Order is the thing distributed systems usually fight about, and we just stopped caring about it. Our broadcast makes no promises about the order two servers see messages in, and now it doesn't have to. No server has to be the authority on anything, so any server can accept any edit for any document, our fleet stays stateless, and we can deploy and autoscale without thinking about it.

Commuting does not mean we can lose messages though. Order doesn't matter, but arrival still does. Every copy of the document has to see every operation eventually, and pub/sub is at most once, so sooner or later it drops one. Nobody gets an error when that happens. The two servers just quietly stop agreeing about what the document says. We will fix that in a minute.

Saving the Document

Everything so far only exists in memory. Restart a server or lose the last client editing a document and it is gone.

The obvious fix is to write to our database on every keystroke, but that is not going to work. A fast typist produces five to ten operations a second, and a busy document has a few of those at once. That is a lot of transactional writes for data that gets overwritten again a few milliseconds later.

What we want is write behind. Edits are applied in memory as they happen, and the expensive database write happens on its own schedule, well behind them.

So we snapshot the document into our real database every so often. Every N operations, or every few seconds of activity, and also immediately when the last client leaves, since that is the moment the document is about to disappear from memory. Loading is the same thing in reverse. When the first client opens a document nobody else is editing, we read the snapshot back and we are ready to go.

The tradeoff is right there in the interval. Whatever was typed since the last snapshot only exists in memory, so a crash takes it with it. Snapshot more often and we lose less but write to the database more. It is a number we should pick on purpose instead of ending up with by accident, and we can tighten it for documents that matter and loosen it for scratch pads.

CRDTs soften this too. Every client is holding its own full copy of the document, so a client that reconnects after a server died can merge what it has back in. The snapshot is our floor, not our only copy.

Catching Up

There is still a hole on the way in.

Carol opens the document and lands on server C. That server has never seen this document before, so its memory is empty, and so is Carol's browser. It subscribes to the channel, which means it now hears every new edit while knowing nothing about the ninety pages that are already there.

So it reads the snapshot out of the database, and this is where write behind comes back around on us. That snapshot is from the last flush. Alice and Bob have been typing for the five seconds since, and none of that is on disk yet. Carol gets the document as it was five seconds ago, and then live edits start landing on it that were written against text she doesn't have.

The current version does exist. It is sitting in memory on server A, where Alice is typing. It just hasn't been written down.

So server C asks for it. It publishes a message on the document's channel saying how much it already has, and server A is subscribed to that channel, so it answers with the rest. No new connections and nothing to discover, we are using the pub/sub that is already there.

What server C sends is called a state vector, and it is tiny. Each person's operations are numbered in order, so "I have Alice's first four hundred and Bob's first thirty" is a couple of numbers.

This is not a compressed copy of the document. It is a bookmark. It says how far along we are and nothing whatsoever about what any of those operations were.

The document itself still has to travel. Server A compares those numbers against its own and sends back the operations that are missing, in full. Cold open and that is the whole document, which was always going to be true. Five seconds behind and it is five seconds of edits instead of ninety pages, which is the case we actually care about.

Carol then does the same thing with server C over her websocket, and everybody is current.

This is our dropped message fix too. A server that missed a broadcast is in exactly the same position server C was in, holding a copy that is quietly behind. Same question, same answer. A lost message stops being permanent and becomes something we recover from.

When Pub/Sub Becomes the Bottleneck

At a big enough scale, the pub/sub is the thing that starts hurting.

Every server subscribed to a document's channel gets every message on that channel, and delivering them is work Redis does on its single thread. The fan out multiplies quickly. Two hundred servers each subscribed to five thousand documents is a million subscriptions on one instance.

It is also all shared, so a single popular document eats Redis time and slows down all of our users. This is the noisy neighbor problem.

Here is what helps, roughly in the order to try it:

Stop publishing on every keystroke. Batch operations over a short window, something like twenty to fifty milliseconds, and publish them as one message. Users cannot feel the difference and it can cut our message volume by an order of magnitude on an active document. This one is easy and it should be the first thing we do.

Watch out for Redis Cluster. This one surprises people. In a normal Redis Cluster, pub/sub messages are broadcast to every node in the cluster, because any client could be subscribed on any node. So adding nodes does not divide our pub/sub load, it multiplies it, and the thing we scaled out to fix gets worse. Redis 7.0 added sharded pub/sub, which sends a channel only to the shard that owns it. Use it, or the cluster migration will be a downgrade.

Consider document affinity. This is sticky sessions, except we stick on the document id instead of the user id. Every document hashes to one server, every connection for it gets routed there, and that server holds all of its connections in local memory without touching the pub/sub at all. The problem is when a pod restarts and we need to hand off the document and all of its connections.

What If Every Document Had Its Own Server?

Let's look back at where all of this actually came from.

The first version worked. One server, a map of documents, a forwarding loop. It was maybe thirty lines and it was correct. Then we added a second server, and from that moment on every single thing in this article has been damage control. The pub/sub, the channel per document, the affinity routing. None of it is about collaborative editing. All of it is about the fact that our users are spread across machines that don't know about each other.

So what if they weren't? What if every document had its own server, and everybody editing that document connected to it?

That solves everything. There is nothing to broadcast, because all the sockets are already in one place. There is nobody to coordinate with, because there is nobody else. We would just go back to the version that worked.

The reason we don't do this is that it sounds insane. A server per document means a server for every meeting note and every grocery list anyone ever typed. That is millions of processes, each holding an operating system's worth of overhead, and almost all of them sitting idle at any given moment. Nobody is provisioning a container per document.

But that objection is about the cost of a server, not about the idea. So what happens if the thing we spin up per document gets small enough and cheap enough?

V8 Isolates

This is what Cloudflare Workers are built on.

Most serverless platforms give each application a container or a micro VM. That is a real machine boundary, which means real memory overhead and a real startup cost, usually somewhere between a few hundred milliseconds and a few seconds. It is why cold starts are something we have to design around.

Workers don't do that. They run our code in a V8 isolate, which is the same mechanism Chrome uses to keep one tab's JavaScript from touching another tab's. An isolate is a sandbox inside a process rather than a process of its own, so a single process can host thousands of them at once. There is no container to pull, no OS to boot, and no runtime to initialize per instance.

The numbers are what matter here. An isolate starts in a couple of milliseconds and carries a few megabytes of overhead instead of a few hundred. Cloudflare can even start ours while the TLS handshake is still finishing, so in practice the cold start disappears into time we were already spending.

That completely changes what "one per document" costs. A million containers is a fantasy. A million isolates, mostly idle, is just Tuesday.

Durable Objects

A Worker on its own is stateless, so it isn't enough for us yet. It has no identity and no memory between requests.

A Durable Object is the piece that adds both. It is a Worker with a name and with storage attached to it, and Cloudflare guarantees that for a given name, exactly one instance of it is alive anywhere in the world at a time. Every request for that name is routed to that one instance, wherever it came from.

So we derive the object's name from the document id. Now every client editing a document connects to the same object, because there is no other object to connect to. We got our server per document.

Look at what that deletes. There is no pub/sub, because all the sockets are already in the same place. There is no load balancer to make sticky and no affinity routing to build, because routing by name is the platform's job. And storage is attached to the object and lives on the same machine, so a snapshot is a local write instead of a background job reaching across the network to a shared database, which means we can afford to do it far more often and lose far less.

Server C never has to ask server A for the difference either, because there is no server C. Carol still has to be handed the ninety pages when she opens the document, but the current version is either in the object's memory or in its own storage, so there is nobody to ask and nothing that can quietly fall behind.

Our whole diagram collapses back into the first version we wrote.

Why This Scales Better

The surprising part is that this is not a step backwards. Going from many servers to one server per document sounds like giving up on scale, and it is the opposite.

Our Redis design has a shared bottleneck in the middle. Every document's traffic goes through the same pub/sub, so documents compete with each other for it, and the way we scale is by splitting that pub/sub into more pieces. With Durable Objects there is nothing in the middle to contend on. Documents are independent by construction, so a million documents is a million tiny objects instead of a million subscriptions on one Redis instance. Adding documents adds capacity instead of consuming it.

That also kills the noisy neighbor problem, since a popular document is now its own object burning its own CPU with no shared pub/sub to slow down for everybody else.

We also stop paying for documents nobody is using. With websocket hibernation, an object that has open connections but no traffic is evicted from memory while those connections stay alive, and it wakes back up when a message arrives. Since starting an isolate costs a couple of milliseconds, waking up is cheap enough to do constantly. Most documents are idle most of the time, and now that costs us nothing.

And because objects are created near whoever first used them, a document a team in Berlin works on lives in Berlin, instead of in whichever region we happened to pick for our cluster.

What It Doesn't Solve

It does not solve concurrent editing for us.

It is tempting to think it does, since the object is single threaded and handles one message at a time, so every edit passes through one place in a definite order. But an order is not a resolution. Alice and Bob both wrote their edit at the same time.

The Tradeoffs

Deleting most of an architecture is not something we get for free. Here is what we are paying for it.

We are marrying a platform. Durable Objects are a Cloudflare product. The programming model, the storage API and the deployment story all come from them, so if we ever have to leave we are not migrating, we are rebuilding.

It is not Node. Workers run in a V8 isolate with web standard APIs, not in a Node process. There is no filesystem, and a good chunk of npm assumes Node built ins that aren't there. There is a compatibility layer that covers a lot of it now, but "does our stack actually run on this" is a question we have to answer before we design around it, not after. For our case we are fine, since a CRDT library is pure JavaScript, but that is luck rather than a general rule.

One object is one thread. This is the same hot document problem as before, except now we cannot route our way out of it. The object is the unit, so a document with a thousand people typing in it has one single threaded process handling all of them, and no amount of scaling helps. It is the noisy neighbor fix seen from the other side, since nothing leaks out of an object and nothing can be spread out of one either. Worth noting that Google Docs caps simultaneous editors at around a hundred and sends everyone else to view only, so this limit is not unique to us. But we should know where our ceiling is instead of discovering it.

One instance is also one point of failure. The guarantee that exactly one object is alive is what makes the whole thing work, and it means there is no replica standing by. While that object is restarting, migrating between machines, or picking up a deploy, that document is unavailable and its sockets drop. It is usually brief, and clients reconnecting cover it, but our availability story for a single document is now "one thing, and it comes back quickly" rather than "any server can serve it".

Location is decided once. The object is created near whoever opened the document first, and that is where it stays. That is great for a team in one city and less great for a team split between Berlin and Sydney, where somebody is always paying the round trip. We can give placement hints, but there is still exactly one location, and the first user picks it.

The cost model is different. We stop paying for idle servers and start paying per object, per request, and for how long objects stay awake. Hibernation makes idle documents genuinely cheap, which is the main thing. But it is a different shape of bill than "we already have some VMs and a Redis", and it is worth modelling against our actual usage before assuming it is cheaper.

None of these are dealbreakers for a collaborative editor. The point is that we traded a pile of infrastructure we control for a primitive we don't, and that is a trade with a direction. It is the right one surprisingly often, but it should be a decision.

Running It Ourselves

The vendor lock in argument took a real hit this month, when Ryan Dahl released celld.

celld is an open source implementation of Workers and Durable Objects that we run on our own machines. It is Apache 2.0, it takes the same Wrangler bundles and config, and it implements the same JavaScript APIs, so the code we wrote for Cloudflare is the code that runs here.

It is also extremely new, and the limitations are real. This is Ryan Dahl's third JavaScript runtime after Node and Deno, So it is very promising.

So this is not what we build our editor on this quarter. But it does change what the lock in tradeoff means. Durable Objects are starting to look less like one vendor's product and more like a shape that other people can implement, and that is a much better position for us to be standing in.

Summary

Let's put the whole thing together.

A stateless websocket fleet behind a load balancer, each server holding its own local sockets and subscribing to a Redis channel per document. Edits are CRDT operations, which commute, so nobody has to be the authority on ordering and every server stays interchangeable. A connecting client tells the server what it already has and gets back the rest, which is also how a server recovers anything the pub/sub dropped. Documents live in memory and get snapshotted on an interval and when the last client leaves, and whatever was typed since then is what a crash costs us. When the pub/sub hurts, we batch, we shard it, and eventually we pin each document to one server.

That last fix is the tell. All of it exists because the sockets for one document are scattered across machines that don't know about each other. Durable Objects remove the scattering instead of compensating for it, and the bill is a platform we don't own and one thread per document.

Why you should cancel your HTTP requests

Omer Davidson — Tue, 15 Apr 2025 20:05:43 +0000

originaly posted at: my personal website

Have you ever built an autocomplete feature that felt slow or returned unexpected results? This often happens because of the way browsers handle HTTP requests. Here's a simple autocomplete example you might have implemented:

import { useState, useEffect } from 'react';

function Autocomplete({ query }) {
  const [results, setResults] = useState([]);

  useEffect(() => {
    try {
        const res = await fetch(`/api/search?q=${query}`);
        const data = await res.json();
        setResults(data);
      } catch (err) {
          console.error(err);
      }
  }, [query]);

  return (
    <ul>
      {results.map(item => (
        <li key={item.id}>{item.name}</li>
      ))}
    </ul>
  );
}

Why This is a Problem

Browsers using HTTP 1.1 typically only allow 6 simultaneous requests to the same domain. With an autocomplete feature, each keystroke triggers a new request. If a user types quickly, you'll quickly hit this limit. This can cause:

Slow performance: Your autocomplete becomes slow because new requests have to wait for previous ones to finish.

Incorrect results (race conditions): Older requests that finish later might overwrite newer, more relevant results.

Simple Solution: Canceling Requests

The good news is, browsers let you cancel ongoing requests. You can easily add this capability to your autocomplete component using an AbortController. Here's how you do it:

import { useState, useEffect } from 'react';

function Autocomplete({ query }) {
  const [results, setResults] = useState([]);

  useEffect(() => {
    const controller = new AbortController();

    async function fetchResults() {
      try {
        const res = await fetch(`/api/search?q=${query}`,
            { signal: controller.signal });
        const data = await res.json();
        setResults(data);
      } catch (err) {
        if (err.name !== 'AbortError') {
          console.error(err);
        }
      }
    }

    fetchResults();

    return () => controller.abort();
  }, [query]);

  return (
    <ul>
      {results.map(item => (
        <li key={item.id}>{item.name}</li>
      ))}
    </ul>
  );
}

Now, each new query cancels the previous request, ensuring only the latest data updates your component. Your autocomplete feature will feel faster and won't have any race conditions.

The HTTP/2 solution

HTTP/2 can alleviate this by multiplexing multiple requests over a single TCP connection, greatly increasing concurrency. However, it's not the default everywhere yet, and many systems still rely on HTTP 1.1.

Even when using HTTP/2 you should still cancel your HTTP requests to avoid race conditions.

Handling Cancellations on Your Server

If you want to go the extra mile, you can also handle canceled requests on your backend.

Here's an example using Express:

import express from 'express';

const app = express();

app.get('/api/search', async (req, res) => {
  const query = req.query.q;
  const dbOperation = startDbSearch(query);

  req.on('close', () => {
    if (!res.writableEnded) {
      dbOperation.cancel();
      console.log('Request canceled by client');
    }
  });

  try {
    const results = await dbOperation;
    res.json(results);
  } catch (err) {
    if (err.message !== 'Operation canceled') {
      res.status(500).send(err.message);
    }
  }
});

app.listen(3000, () => {
  console.log('Server running on port 3000');
});

By stopping expensive operations like DB queries you can reduce server load.

understanding and implementing request cancellation, you significantly enhance the responsiveness and efficiency of your web applications.

5 easy ways to elevate your website

Omer Davidson — Wed, 11 Dec 2024 20:49:29 +0000

I see these 5 mistakes made all the time, even by experienced developers. And they are so easy to fix once you start to look out for them.

1. Prevent layout shifts on overflow

If you have an element with overflow: auto then it will have a scrollbar only when the element is overflowing. The problem is that once the element is overflowing and the scrollbar appears, the content has shrunk to accommodate the width of the scrollbar.

To avoid the unnecessary layout shift, add:
scrollbar-gutter: stable

It will reserve space for the scrollbar even if it is not visible.

At the time of writing this, only 74% of users have this feature. But it is a nice progressive enhancement. Meaning those on newer browser can enjoy a better user experience while those that are on older browsers are not affected.

2. respect the device's preference for dark mode

If you already implemented dark mode in your website, you can spare the user from selecting dark mode manually by checking it's device's preference for light or dark mode

if (window.matchMedia && window.matchMedia('(prefers-color-scheme: dark)').matches) {
    // dark mode
}

So if the user already selected dark mode in it's settings, then you can have the default value be dark mode in your website as well.

Some websites even choose to not have a dark mode toggle at all and rely solely on the device's preference.

3. Links should be real links

When you have a button that should redirect to a different part of your website, the obvious way to do it is to have an event listener for click and to redirect the user using JavaScript.
This is wrong, whenever you can use a browser primitive (eg: link, form) then you should almost always use that instead.

using <a> tag instead has a lot of benefits.

Being able to Ctrl click (or long press on mobile) to open the link in a different tab
Hovering over the link shows you where you will be redirected to
Other programs like screen readers and search engine crawlers will work a lot better

If you are using react-router or Next.js than you should use that framework's Link component to prevent full page reloads.

4. Link previews

When a user shares a link to your website, all chat and social media apps have a preview feature to see some of the content of that website before the user clicks it. By simply adding a couple of meta tags to your <head> object you will allow other apps to show previews to your website.

<head>
<title>The Rock (1996)</title>
<meta property="og:title" content="The Rock" />
<meta property="og:type" content="video.movie" />
<meta property="og:url" content="https://www.imdb.com/title/tt0117500/" />
<meta property="og:image" content="https://ia.media-imdb.com/images/rock.jpg" />
...
</head>

And in react 19 it is easier than ever to edit your <head> object. You previously had to use third party libraries like react-helmet but now it is built in to react.

function BlogPost({ post }) {
  return (
    <article>
      <h1>{post.title}</h1>
      <p>{post.content}</p>

      <title>{post.title}</title>
      <meta name="description" content={post.excerpt} />
      <meta property="og:title" content={post.title} />
      <meta property="og:description" content={post.excerpt} />
      <meta property="og:image" content={post.image} />
      <meta property="og:url" content={post.url} />
      <meta property="og:type" content="article" />

    </article>
  );
}

This is called the Open Graph Protocol.
A very useful tool is social share preview which lets you test how your preview will look in different websites and give you advice on how to make it better.

Note that this works best with server side rendering as most social media apps won't run JavaScript to get your link preview.

5. Use lables with your inputs

I often see checkboxes and when I try to click the checkbox's label nothing happens. Aside from poor accessibility, this means the user has to click the tiny checkbox to select it.

To solve this use the <label> tag.

<label>
    <input type="checkbox"> Click me to select the checkbox
</label>

This works for all input types. Clicking the label of a text input for example will focus into the textbox.

A brief history of ES modules

Omer Davidson — Sat, 16 Nov 2024 18:47:20 +0000

You've probably used ES modules in modern JavaScript development, but do you know the history behind their evolution? Understanding the journey from early JavaScript practices to today’s module system will help you appreciate how far we've come and why ES modules are a game changer.

The Early Days: JavaScript's Beginnings

The year is 1995, four years after the first web page was created. Most websites were simple — static pages with text and minimal interactivity. However, developers were quickly looking for ways to make web pages more dynamic.

In this environment, Netscape (the dominant web browser at the time) hired Brendan Eich to create a scripting language that would run directly in the browser. This led to the birth of JavaScript, a language designed to be simple and accessible, especially for non-programmers like web designers. And so he did, completing the first version in 10 days.

Originally, JavaScript was meant to add small enhancements to web pages, like form validation, without needing to round-trip data to the server. However, as websites became more interactive, JavaScript quickly grew beyond its original purpose.

The Global Scope and Early Workarounds

In the early days, all JavaScript code lived in the global scope. As more developers added code to the same page, the risk of name collisions grew. If multiple scripts used the same variable or function name, the code could break in unexpected ways.

To manage this, developers resorted to naming conventions to prevent collisions. If a piece of code was meant to run only once, developers often wrapped it in an IIFE (Immediately Invoked Function Expression). This kept functions and variables scoped within the function, preventing them from polluting the global namespace.

(function init() {
    function getData(){
        // ...
     }
})()

This was good enough at the time as most websites were server side rendered with very little client side logic.

CommonJS: Server-side modules

In 2008 Ryan Dahl created Node.js, a JavaScript runtime for building server side applications. This opened up a whole new world of possibilities, but the lack of a module system meant that developers struggled to manage large codebases.

In 2009, CommonJS was introduced to solve this problem for the server-side. The CommonJS module system allowed developers to define modules, expose functionality, and import other modules. Here's an example of how it worked:

const math = require("./math");

function subtract(a,b) {
    return math.add(a,-b);
}

module.exports = {
    subtract: subtract
}

With CommonJS, each file is treated as its own module, and modules are imported using the require function and exported using module.exports.

Some key features of CommonJS include:

The file extension is optional when requiring a module (e.g., require('./math') automatically looks for math.js).
Module loading is synchronous, meaning the program waits for the module to load before continuing execution.

Later in the article we will see why Ryan Dahl admitted to regretting these 2 design decisions.

AMD: Optimizing the Browser

Around the same time, another module system called AMD (Asynchronous Module Definition) was developed. While CommonJS was primarily focused on server-side JavaScript, AMD was designed to handle client-side JavaScript in the browser.

The key feature of AMD was its ability to load modules asynchronously. This allowed the browser to load only the JavaScript needed for a page at any given time, improving performance by reducing initial page load time. It also solved issues related to dependency resolution, ensuring that a module would only run once its dependencies had finished loading.

AMD's benefits included:

Smaller JavaScript files loaded on-demand.
Fewer page load errors due to more predictable module loading.
Performance optimization through asynchronous loading.

With the rise of npm in 2010 (a package manager for server-side JavaScript), the need for sharing code across the browser and the server became apparent. Enter Browserify, a tool that allowed developers to use CommonJS modules in the browser by transforming the code to be compatible with the browser’s environment.

UMD: The Best of Both Worlds... Sort Of

With the 2 competing standards, CommonJS and AMD. There was a need for a single module system that could work everywhere without the need for a build step. And in 2011, the universal module definition(UMD) was introduced.

UMD combined the best of both worlds, allowing developers to write a module that could run in:

Node.js (using CommonJS).
Browsers (using AMD).
The global scope (if neither CommonJS nor AMD was present).

UMD became very popular amongst library authors with notable libraries like Lodash, Underscore.js, Backbone.js, and Moment.js adpoting it. However, UMD had some significant downsides. While it solved the compatibility problem, it came with the complexity of managing both systems, and it inherited the problems of both AMD and CommonJS.

ES modules: The ultimate modules standard

In 2015, ES Modules (ESM) were introduced as part of the ECMAScript standard, finally offering a native module system for JavaScript. By 2017, all major browsers supported ES modules, and in 2020, Node.js added support as well.

Let's see why ES modules are the best:

1. Convenient Syntax

The following UMD code:

(function (root, factory) {
  if (typeof define === 'function' && define.amd) {
    // AMD
    define([], factory);
  } else if (typeof module === 'object' && module.exports) {
    // CommonJS
    module.exports = factory();
  } else {
    // Global
    root.add = factory();
  }
}(this, function () {
  function add(a, b) {
    return a + b;
  }

  return add;
}));

can now be reduced to:

export function add(a, b) {
  return a + b;
}

To be fair, no one actually wrote UMD like that. They used tools like umdify to generate that code. But with ES modules being built in we can skip the build step and have a smaller bundle size.

2. Better Optimization

ES modules are static, meaning that tools can analyze the code structure at compile time to determine which code is being used and which is not. This allows for tree shaking, where unused code is removed from the final bundle.

Because CommonJS and AMD modules are dynamic (evaluated at runtime), tree shaking is much less efficient with these systems, often resulting in larger bundles.

3. More Explicit Code

When importing a module with CommonJS, specifiying the file extension is optional.

// load math.js file
const math = require('./math')

But what does math actually refer to? Is it a JavaScript file? A JSON file? An index.js file inside a math directory?

When using static analysis tools like ES Lint,typescript or prettier every require becomes a guessing game.
is it math.js?
is it math.jsx?
is it math.cjs?
is it math.mjs?
is it math.ts?
is it math.tsx?
is it math.mts?
is it math.cts?
is it math/index.js?
is it math/index.jsx?

You get the idea.

Reading a file is expensive. It is a lot less performant than reading from memory. Importing math/index.js results in 9 IO operations instead of 1! And this guessing game is slowing down our tooling and hurts developer experience.

In ES modules we avoid this mess by having file extensions be mandatory.

4. Asynchronous Loading

Unlike CommonJS, which loads modules synchronously (blocking the entire process until the module is loaded), ES modules are asynchronous. This allows JavaScript to continue executing while the module is being loaded in the background, improving performance — particularly in environments like Node.js.

Migration Challenges: Why It Took So Long

Despite the clear benefits, adopting ES modules was no simple task. Here’s why the transition took so long:

1. Costly Migration

Switching from CommonJS to ES modules was not a trivial change, especially for large projects. The syntax differences, combined with the need for tooling support, made migration a significant effort.

2. Lack of Node.js Support

It took node.js 5 years to fully support ES modules. During this time, developers had to maintain compatibility with both CommonJS (on the server) and ES modules (on the browser). This dual support created a lot of friction in the ecosystem.

3. Compatability issues

Even after Node.js added support for ES modules, CommonJS modules couldn’t load ES modules. While ES modules could load CommonJS modules, the two systems weren’t fully interoperable, creating additional headaches for package authors who had to support both systems.

The Future: ES Modules Are Here to Stay

The future of JavaScript modules is bright, and here are a few key developments that will make ES modules the dominant system moving forward:

1. Node.js 23

In Node.js 23 we finally have the ability to load ES modules from CommonJS.
There is a small caveat: ES module that use top-level await cannot be imported into CommonJS, since await can only be used in asynchronous functions, and CommonJS is synchronous.

2. JSR (JavaScript Registry)

A new javascript package registry that competes with npm. It has a lot of advantages over npm which I won't go over here. But the interesting thing is you are only allowed to upload ES modules packages. No need for supporting the old standards.

Conclusion

The journey from global scope hacks to modern ES modules has transformed how we structure JavaScript. After years of experimenting with CommonJS, AMD, and UMD, ES modules have emerged as the clear standard, offering simpler syntax, better optimization, and improved performance.

While migrating to ES modules has been challenging, especially with Node.js support and ecosystem compatibility, the benefits are undeniable. With Node.js 23 improving interoperability and new tools like JSR promoting a unified module system, ES modules are set to become the default for JavaScript.

As we continue to embrace ES modules, we can look forward to cleaner, faster, and more maintainable code, marking a new era of modularity in JavaScript development.

A useState performance tip you may not have known

Omer Davidson — Tue, 29 Oct 2024 18:11:30 +0000

Let's say we have a react component with a useState inside it.

const expensiveCalculation = () => {
  // ...
}

export default function AboutPage() {
  const [state, setState] = useState(expensiveCalculation())

    return (
      // ...
    );
  }

We initiate the state with the result of the expensiveCalculation function.
Every time the component re-renders, the function expensiveCalculation will run even though we only need it's result as the initial value of useState. The function's result will not be used.

To avoid the expensive calculation during re-renders, pass the function itself without calling it. react is smart enough to invoke the function itself on mount and not every render.

const expensiveCalculation = () => {
  // ...
}

export default function AboutPage() {
  const [state, setState] = useState(expensiveCalculation)

    return (
      // ...
    );
  }