Real-Time State Synchronization: Building Responsive Web Applications with OT, CRDTs, and Optimistic Updates

#programming #devto #webdev #softwareengineering

As a best-selling author, I invite you to explore my books on Amazon. Don't forget to follow me on Medium and show your support. Thank you! Your support means the world!

When you use a web application today, you expect it to react instantly. You add an item to a cart, and the number updates. You type in a document, and the text appears. Behind this simple experience is a complex problem: keeping the state of the application—the data—consistent across your device, the server, and potentially every other user's screen. This is state synchronization. I want to talk about the practical ways we solve this.

Think of a collaborative document. Two people type at the same time. Without a system, their letters would clash and overwrite each other, creating a mess. Operational Transformation (OT) is a method that untangles this. It works by taking each user's edit—like "insert 'A' at position 5"—and mathematically transforming it against other concurrent edits. This ensures that no matter what order the edits arrive in, everyone ends up with the same final document.

Let me show you what this looks like in code. We'll create a simple text document that can handle operations from multiple clients.

class TextDocument {
  constructor(content = '') {
    this.content = content;
    this.version = 0;
    this.pendingOperations = [];
  }

  applyOperation(operation, clientVersion) {
    let transformedOp = operation;

    for (const pendingOp of this.pendingOperations) {
      if (pendingOp.clientId !== operation.clientId) {
        transformedOp = this.transformOperation(transformedOp, pendingOp);
      }
    }

    this.content = this.applyToContent(this.content, transformedOp);
    this.version++;

    this.pendingOperations.push({
      ...operation,
      appliedAt: this.version
    });

    this.pendingOperations = this.pendingOperations.filter(
      op => op.appliedAt > this.version - 10
    );

    return { content: this.content, version: this.version };
  }

  transformOperation(op1, op2) {
    if (op1.type === 'insert' && op2.type === 'insert') {
      if (op1.position < op2.position) {
        return op1;
      } else {
        return { ...op1, position: op1.position + op2.text.length };
      }
    }
    return op1;
  }

  applyToContent(content, operation) {
    switch (operation.type) {
      case 'insert':
        return content.slice(0, operation.position) + 
               operation.text + 
               content.slice(operation.position);
      case 'delete':
        return content.slice(0, operation.position) + 
               content.slice(operation.position + operation.length);
      default:
        return content;
    }
  }
}

On the client side, we manage sending local changes and handling updates from the server.

class CollaborativeClient {
  constructor(documentId) {
    this.documentId = documentId;
    this.localContent = '';
    this.localVersion = 0;
    this.outgoingOperations = [];
    this.ws = new WebSocket(`wss://api.example.com/docs/${documentId}`);
    this.setupWebSocket();
  }

  applyLocalChange(operation) {
    this.localContent = this.applyOperation(this.localContent, operation);

    this.outgoingOperations.push({
      ...operation,
      clientId: this.clientId,
      clientVersion: this.localVersion
    });

    this.ws.send(JSON.stringify({
      type: 'operation',
      operation,
      version: this.localVersion
    }));

    this.localVersion++;
  }

  handleServerOperation(serverOp) {
    this.outgoingOperations = this.outgoingOperations.map(op => 
      this.transformOperation(op, serverOp)
    );

    this.localContent = this.applyOperation(this.localContent, serverOp);
    this.localVersion = serverOp.serverVersion;
  }
}

OT is powerful for real-time collaboration, but it often requires a central server to manage the transformation logic. What if we need to work completely offline, or in a peer-to-peer setting? This is where Conflict-free Replicated Data Types (CRDTs) shine.

CRDTs are data structures designed to be merged. You can have two independent copies, make changes to each, and later merge them together with a guarantee that they will become identical. There is no need for a central coordinator. This makes them perfect for features like a shared to-do list that syncs once your phone reconnects to the internet.

A simple CRDT is a Grow-Only Set. You can only add items, never remove them. Merging two sets is just taking their union.

class GSet {
  constructor(elements = new Set()) {
    this.elements = new Set(elements);
  }

  add(element) {
    this.elements.add(element);
    return this;
  }

  has(element) {
    return this.elements.has(element);
  }

  merge(other) {
    return new GSet(new Set([...this.elements, ...other.elements]));
  }

  compare(other) {
    return [...this.elements].every(el => other.has(el));
  }
}

For data that can change, like a user's display name, we might use a Last-Writer-Wins Register. It stores a value along with a timestamp. When merging, the value with the most recent timestamp wins.

class LWWRegister {
  constructor(value = null, timestamp = Date.now(), replicaId = '') {
    this.value = value;
    this.timestamp = timestamp;
    this.replicaId = replicaId;
  }

  set(value, replicaId) {
    const now = Date.now();
    return new LWWRegister(value, now, replicaId);
  }

  merge(other) {
    if (this.timestamp > other.timestamp) {
      return this;
    } else if (this.timestamp < other.timestamp) {
      return other;
    } else {
      return this.replicaId > other.replicaId ? this : other;
    }
  }

  get() {
    return this.value;
  }
}

We can build a more useful structure, like an observable map, from these basic CRDTs.

class CRDTMap {
  constructor(replicaId) {
    this.replicaId = replicaId;
    this.data = new Map();
    this.listeners = new Set();
  }

  set(key, value) {
    const newRegister = new LWWRegister(value, Date.now(), this.replicaId);
    const current = this.data.get(key);

    if (current) {
      this.data.set(key, current.merge(newRegister));
    } else {
      this.data.set(key, newRegister);
    }

    this.notifyListeners();
    return this;
  }

  get(key) {
    const register = this.data.get(key);
    return register ? register.get() : undefined;
  }

  merge(otherMap) {
    for (const [key, otherRegister] of otherMap.data) {
      const current = this.data.get(key);

      if (current) {
        this.data.set(key, current.merge(otherRegister));
      } else {
        this.data.set(key, otherRegister);
      }
    }

    this.notifyListeners();
    return this;
  }

  subscribe(listener) {
    this.listeners.add(listener);
    return () => this.listeners.delete(listener);
  }

  notifyListeners() {
    for (const listener of this.listeners) {
      listener(this.toJSON());
    }
  }
}

CRDTs give us eventual consistency, but sometimes we need to understand the history of changes. We need to know if one change happened before another, especially when resolving conflicts. This is where version vectors come in.

A version vector is a logical clock. Instead of one number, it keeps a counter for every "replica" or source of changes (like a user's device). If my phone makes a change, my phone's counter in the vector goes up. This lets us compare versions and see which one is newer, or if they happened concurrently.

class VersionVector {
  constructor(entries = new Map()) {
    this.entries = new Map(entries);
  }

  increment(replicaId) {
    const current = this.entries.get(replicaId) || 0;
    this.entries.set(replicaId, current + 1);
    return this;
  }

  merge(other) {
    const merged = new Map(this.entries);

    for (const [replicaId, counter] of other.entries) {
      const current = merged.get(replicaId) || 0;
      merged.set(replicaId, Math.max(current, counter));
    }

    return new VersionVector(merged);
  }

  compare(other) {
    let thisGreater = false;
    let otherGreater = false;

    const allReplicas = new Set([
      ...this.entries.keys(),
      ...other.entries.keys()
    ]);

    for (const replicaId of allReplicas) {
      const thisCounter = this.entries.get(replicaId) || 0;
      const otherCounter = other.entries.get(replicaId) || 0;

      if (thisCounter > otherCounter) {
        thisGreater = true;
      } else if (thisCounter < otherCounter) {
        otherGreater = true;
      }
    }

    if (thisGreater && !otherGreater) {
      return 'greater';
    } else if (!thisGreater && otherGreater) {
      return 'less';
    } else if (thisGreater && otherGreater) {
      return 'concurrent';
    } else {
      return 'equal';
    }
  }
}

We can use this to build a store that knows how to sync intelligently.

class VersionedStore {
  constructor(replicaId) {
    this.replicaId = replicaId;
    this.data = new Map();
    this.versions = new Map();
  }

  put(key, value) {
    const currentVersion = this.versions.get(key) || new VersionVector();
    const newVersion = currentVersion.increment(this.replicaId);

    this.data.set(key, value);
    this.versions.set(key, newVersion);

    return { value, version: newVersion };
  }

  sync(otherStore) {
    const conflicts = [];

    for (const [key, otherValue] of otherStore.data) {
      const otherVersion = otherStore.versions.get(key);
      const currentVersion = this.versions.get(key);

      if (!currentVersion) {
        this.data.set(key, otherValue);
        this.versions.set(key, otherVersion);
      } else {
        const comparison = currentVersion.compare(otherVersion);

        switch (comparison) {
          case 'less':
            this.data.set(key, otherValue);
            this.versions.set(key, otherVersion);
            break;
          case 'greater':
            break;
          case 'concurrent':
            conflicts.push({
              key,
              ourValue: this.data.get(key),
              theirValue: otherValue
            });
            const mergedVersion = currentVersion.merge(otherVersion);
            this.versions.set(key, mergedVersion);
            break;
        }
      }
    }
    return conflicts;
  }
}

Sometimes, the data we sync is large, like a full document. Sending the entire document on every change wastes bandwidth. Differential synchronization is about sending only what changed.

The core idea is to keep three copies: the current version on the server, the last known server version on the client (called a "shadow"), and the client's working copy. To sync, the client calculates the difference between its shadow and its working copy, sends that "patch" to the server, and gets a patch back.

class DiffSync {
  constructor(content = '') {
    this.content = content;
    this.shadow = content;
    this.backup = content;
  }

  generatePatch() {
    const diff = this.computeDiff(this.shadow, this.content);
    return this.diffToPatch(diff);
  }

  applyPatch(patch) {
    const newShadow = this.applyPatchToText(this.shadow, patch);
    const diff = this.computeDiff(this.backup, newShadow);
    const patchToContent = this.diffToPatch(diff);

    this.content = this.applyPatchToText(this.content, patchToContent);
    this.shadow = newShadow;
    this.backup = this.content;

    return this.content;
  }

  computeDiff(text1, text2) {
    // Implementation of a diff algorithm (e.g., Myers)
    // Returns a list of edits: [['equal', 'abc'], ['insert', 'x'], ['delete', 'y']]
    // Simplified here for brevity.
    const diffs = [];
    // ... complex diff calculation logic ...
    return diffs;
  }

  diffToPatch(diff) {
    const patch = [];
    let position = 0;

    for (const [type, text] of diff) {
      if (type === 'equal') {
        position += text.length;
      } else if (type === 'insert') {
        patch.push({ type: 'insert', position, text });
        position += text.length;
      } else if (type === 'delete') {
        patch.push({ type: 'delete', position, length: text.length });
      }
    }
    return patch;
  }
}

All these patterns handle the backend sync logic. But from a user's perspective, the most important thing is that the UI feels fast. This is where optimistic updates come in. The principle is simple: don't wait for the server to confirm a change. Update the UI immediately. If the server later rejects the change, roll it back and show an error.

This gives the feeling of instant response, which is crucial for a good experience. It does, however, require careful state management.

class OptimisticUpdateManager {
  constructor() {
    this.localState = {};
    this.originalState = {};
    this.pendingUpdates = new Map();
    this.updateId = 0;
    this.listeners = new Set();
  }

  applyUpdate(key, updater) {
    const updateId = ++this.updateId;

    if (!this.originalState.hasOwnProperty(key)) {
      this.originalState[key] = this.localState[key];
    }

    const previousValue = this.localState[key];
    const newValue = updater(previousValue);
    this.localState[key] = newValue;

    const update = {
      id: updateId,
      key,
      previousValue,
      newValue,
      status: 'pending'
    };

    this.pendingUpdates.set(updateId, update);
    this.notifyListeners(key, newValue);

    this.sendToServer(update);

    return updateId;
  }

  confirmUpdate(updateId, serverValue) {
    const update = this.pendingUpdates.get(updateId);
    if (!update) return;

    update.status = 'confirmed';

    if (serverValue !== undefined && serverValue !== update.newValue) {
      this.localState[update.key] = serverValue;
      this.notifyListeners(update.key, serverValue);
    }

    this.pendingUpdates.delete(updateId);
  }

  rejectUpdate(updateId, error, serverValue = null) {
    const update = this.pendingUpdates.get(updateId);
    if (!update) return;

    update.status = 'rejected';
    update.error = error;

    if (serverValue !== null) {
      this.localState[update.key] = serverValue;
      this.notifyListeners(update.key, serverValue);
    } else {
      this.localState[update.key] = update.previousValue;
      this.notifyListeners(update.key, update.previousValue);
    }

    this.pendingUpdates.delete(updateId);
  }

  subscribe(listener) {
    this.listeners.add(listener);
    return () => this.listeners.delete(listener);
  }

  notifyListeners(key, value) {
    for (const listener of this.listeners) {
      listener(key, value);
    }
  }
}

In a React application, you might use this with a custom hook.

function useOptimisticUpdate(initialState) {
  const [state, setState] = useState(initialState);
  const managerRef = useRef(new OptimisticUpdateManager());

  useEffect(() => {
    const manager = managerRef.current;
    const unsubscribe = manager.subscribe((key, value) => {
      setState(prev => ({ ...prev, [key]: value }));
    });
    return unsubscribe;
  }, []);

  const applyUpdate = useCallback((key, updater) => {
    return managerRef.current.applyUpdate(key, updater);
  }, []);

  return {
    state,
    applyUpdate,
    manager: managerRef.current
  };
}

So, how do you choose? Think about your app's needs. If you're building a Google Docs-style editor, you'll likely use Operational Transformation. For a note-taking app that works offline, a CRDT-based approach is robust. If you're syncing large files or text documents infrequently, differential synchronization saves bandwidth. For almost any interactive app, optimistic updates are essential for perceived performance. Version vectors are your tool for understanding causality when you need to merge histories.

These patterns are not magic. They are careful, mathematical answers to the messy problem of keeping things in sync across a network. They allow us to build applications that feel simple and immediate, even when the machinery underneath is coordinating data across the globe. I find that understanding them turns a daunting architectural challenge into a set of clear, implementable choices. You can pick the right tool for the job and build something that feels seamless to the person using it.

📘 Checkout my latest ebook for free on my channel!

Be sure to like, share, comment, and subscribe to the channel!

101 Books

101 Books is an AI-driven publishing company co-founded by author Aarav Joshi. By leveraging advanced AI technology, we keep our publishing costs incredibly low—some books are priced as low as $4—making quality knowledge accessible to everyone.

Check out our book Golang Clean Code available on Amazon.

Stay tuned for updates and exciting news. When shopping for books, search for Aarav Joshi to find more of our titles. Use the provided link to enjoy special discounts!