📝 Designing a Scalable Document Convergence System (Used by Big Tech)

How do systems like Google Docs and Figma allow multiple people to edit the same document simultaneously, without conflicts or data loss? Let's dive deep into document convergence system design—covering core algorithms, architecture, and real-world implementations from big tech.

Operational Transformation (OT)
Conflict-free Replicated Data Types (CRDT)

📌 Problem Definition

Goal: Build a system where multiple users can edit a document concurrently, and all changes eventually converge to a consistent state.

✅ Key Requirements

Functional

Concurrent editing by multiple users
Real-time updates (low latency)
Conflict resolution & convergence
Undo/redo operations
Document history/versioning

Non-Functional

High availability & fault tolerance
Scalable to millions of documents/users
Offline support (for CRDTs)
Secure access control

🧠 Core Algorithms for Document Convergence

1. Operational Transformation (OT)

Used by: Google Docs

Core Idea: Transform operations (insert/delete) against concurrent ones to preserve user intention.
Properties:
- Convergence
- Causality preservation
- Intention preservation

Example: If two users insert text at the same index, OT reorders based on timestamps/user priority.

📚 Read:

2. Conflict-free Replicated Data Types (CRDTs)

Used by: Figma, Notion, Automerge

Core Idea: Each client operates on their own replica. Changes are merged using mathematically designed structures (e.g., LWW registers, RGA lists).
Types:
- G-Counter / PN-Counter
- OR-Set
- RGA (Replicated Growable Array) for text

Advantages:

Offline-first
Peer-to-peer support
Simpler merging without central coordination

📚 Read:

🏢 Big Tech Implementations

Company	Algorithm	Notable Features	Reference
Google Docs	OT	Central server transforms ops, real-time sync	Wave Protocol
Figma	CRDT (Custom)	Vector clocks, peer-to-peer design	Figma Multiplayer
Notion	CRDT (Yjs)	Offline editing, real-time sync via WebSockets	Yjs CRDT
Dropbox Paper	OT	Centralized transformation service	Dropbox Engineering
Automerge	CRDT	JSON CRDT for offline-friendly apps	Automerge

🏗️ System Architecture

Real-Time Collaborative Editor System

Clients (Web/Mobile)
    ↕ (WebSocket / WebRTC)
Collaboration Server
    ↳ Change Processor (CRDT / OT)
    ↳ Document State Store (Redis/In-Memory)
    ↳ Event Queue (Kafka / Redis Streams)
    ↳ Persistent Store (PostgreSQL / S3 / Cassandra)

Components

Editor (Client):
- Uses local data model (CRDT/OT)
- Applies changes optimistically
- Listens for remote updates
Collaboration Server:
- Applies merge algorithm
- Broadcasts updates to other users
- Maintains real-time presence
Storage:
- Event sourcing for operation logs
- Snapshots for fast loading
- Version control for history

⚙️ Technology Stack

Component	Tech Options
Editor	Quill.js, Slate.js, ProseMirror, Lexical
Real-time Sync	WebSockets, WebRTC, MQTT
CRDT	Yjs, Automerge, Logoot, LSEQ
OT	ShareDB, Apache Wave
State Store	Redis, Memcached
Persistent DB	PostgreSQL, Cassandra, DynamoDB, S3
Streaming	Kafka, Redis Streams
Auth	OAuth2, JWT

⚠️ Challenges & Trade-offs

Challenge	OT	CRDT
Real-time Editing	✅ Stable	✅ Stable
Offline Support	❌ Complex	✅ Built-in
Merge Complexity	🔺 High	✅ Simpler
Storage Overhead	Medium	High (metadata-heavy)
Peer-to-Peer Support	❌ Server-centric	✅ Works well
Integration Ease	Mature but complex	Modern & evolving

📦 Example CRDT: Yjs + WebSocket + Redis

Yjs on frontend to manage local document
WebSocket to sync changes
Redis pub/sub to broadcast changes to all clients
Snapshot to PostgreSQL/S3 every X minutes

📚 Yjs WebSocket Server

🏁 Conclusion

Document convergence is a complex, foundational problem in collaborative software. Big tech companies rely on Operational Transformation and CRDTs to ensure that users editing documents simultaneously see a consistent, coherent result.