A comprehensive system design walkthrough — HLD + LLD
Introduction
Weave is an open-source video conferencing platform that solves a fundamental problem: internet quality should never compromise recording quality. Unlike Zoom or Google Meet, which record server-side (and lose quality on bad connections), Weave records locally on each participant's device in encrypted 60-second chunks, then intelligently merges and processes them in the cloud.
The platform includes:
- Real-time video conferencing (powered by Jitsi Meet)
- Local chunk-based recording with AES-256-GCM encryption
- Per-meeting Content Encryption Keys wrapped with RSA-OAEP-256
- An in-browser video editor with timeline, transitions, overlays, and effects
- Multi-camera (multicam) editing with speaker detection and auto-cut
- HLS adaptive streaming with 4 quality profiles
- Scheduling with Google Calendar, Slack, Discord integrations
Tech Stack: React 19, Express 5, PostgreSQL, Redis, FFmpeg, Bun, Kubernetes, ArgoCD
High-Level Design (HLD)
System Architecture
Weave follows a microservices architecture within a monorepo (Turborepo). Six services communicate via HTTP REST, WebSocket, and Redis-backed asynchronous job queues.
┌──────────────────────────────────────────────────────────────────────────┐
│ CLIENT (React + Vite) │
│ Dashboard │ Live Meeting (Jitsi) │ Video Editor │ Recording Viewer │
└────────────┬─────────────────────────────────────────────────────────────┘
│ HTTPS │ WebSocket
▼ ▼
┌────────────────────────────┐ ┌──────────────────────────────┐
│ BACKEND (Express 5) │ │ WS-RELAYER (Bun WebSocket) │
│ Port 3000 │ │ Port 9093 │
│ REST API ─ Auth │ │ ─ Chat messaging │
│ ─ Meetings ─ Recording │ │ ─ Participant presence │
│ ─ Editor ─ Notifications │ │ ─ Recording state events │
│ ─ GitHub ─ Google OAuth │ │ ─ Typing indicators │
└──────┬─────────────────────┘ └──────────┬───────────────────┘
│ │
▼ ▼
┌──────────────────────────────────────────────────────────────────────┐
│ INFRASTRUCTURE │
│ ┌──────────┐ ┌────────────────────┐ ┌──────────────────────────┐ │
│ │PostgreSQL│ │ Redis │ │ S3-Compatible Storage │ │
│ │(NeonDB) │ │ ─ Job Queues (5) │ │ (AWS S3 / GCS) │ │
│ │Prisma │ │ ─ Rate Limiting │ │ ─ Encrypted chunks │ │
│ │14 Models │ │ ─ Chat History │ │ ─ Final recordings │ │
│ │15 Enums │ │ ─ CEK Storage │ │ ─ HLS streams │ │
│ └──────────┘ └────────────────────┘ │ ─ Editor assets │ │
│ └──────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────┘
│
┌──────────────────┬──────────────────────┐
▼ ▼ ▼
┌──────────────┐ ┌───────────────┐ ┌──────────────────────┐
│Merger-Worker │ │Editor-Worker │ │ Transcoder │
│(Bun + FFmpeg)│ │(Bun + FFmpeg) │ │(Bun + FFmpeg) │
│ │ │ │ │ │
│Decrypts & │ │Renders video │ │Converts MP4 → HLS │
│merges chunks │ │editor exports│ │360p/480p/720p/1080p │
│→ per-user │ │with effects, │ │+ poster + sprites │
│videos → grid │ │transitions, │ │+ VTT thumbnails │
│video │ │overlays │ │ │
└──────────────┘ └───────────────┘ └──────────────────────┘
Redis Queue Architecture
Five job queues power asynchronous processing. All workers use Redis Lists with BLPOP for blocking dequeue and RPUSH for enqueue.
| Queue | Producer | Consumer | Purpose |
|---|---|---|---|
ProcessVideo |
Backend (on meeting end) | Merger-Worker | Decrypt & merge chunks |
TranscodeVideo |
Merger-Worker, Editor-Worker | Transcoder | HLS conversion |
EditorRender |
Backend (on export) | Editor-Worker | Render video exports |
Notifications |
All services | Backend | Deliver user notifications |
MeetingInvitations |
Backend | Backend | Send meeting invites |
Deployment Architecture
Weave runs on Kubernetes managed via ArgoCD (GitOps). All services are deployed in the weave namespace.
Internet
│
┌───────┴────────┐
│ NGINX Ingress │ (cert-manager TLS)
└───────┬────────┘
│
┌──────────────────┼──────────────────┐
▼ ▼ ▼
weave.krishlabs.tech api.weave... wss.weave...
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Frontend │ │ Backend │ │ WS-Relayer │
│ (React SPA) │ │ Express 5 │ │ Bun WS │
│ Pod: 1 │ │ Pod: 1 │ │ Pod: 1 │
│ CPU: 300m │ │ CPU: 500m │ │ CPU: 300m │
│ RAM: 256Mi │ │ RAM: 512Mi │ │ RAM: 256Mi │
└──────────────┘ └──────────────┘ └──────────────┘
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│Merger-Worker │ │Editor-Worker │ │ Transcoder │
│ Pod: 1 │ │ Pod: 1 │ │ Pod: 1 │
│ CPU: 512m │ │ CPU: 512m │ │ CPU: 512m │
│ RAM: 1Gi │ │ RAM: 1Gi │ │ RAM: 1Gi │
│ Vol: 20Gi │ │ Vol: 20Gi │ │ Vol: 20Gi │
└──────────────┘ └──────────────┘ └──────────────┘
┌──────────────────┐
│ Redis (in-cluster) │
│ Pod: 1 │
│ CPU: 100m, RAM: 128Mi │
└──────────────────┘
External Dependencies:
- NeonDB (PostgreSQL) — managed database on AWS via Neon
- S3-compatible storage — production bucket on krishlabs.tech
- Let's Encrypt — TLS certs via cert-manager
Core Data Flow: The Recording Pipeline
This is the heart of Weave's innovation:
1. MEETING START
Client generates 256-bit Content Encryption Key (CEK)
Fetches server's RSA-4096 public key
Wraps CEK with RSA-OAEP-256
Stores wrapped CEK → POST /keys/meeting/:id/wrapped-cek
Redis stores it (7-day TTL)
2. EVERY 60 SECONDS (during meeting)
Capture MediaStream → MediaRecorder → WebM chunk
Generate 96-bit IV (deterministic: SHA-256 of userId:seqNum)
Encrypt: AES-256-GCM(chunk, CEK, IV)
Upload: POST /upload-chunk
└─ Backend stores to S3 + upserts MediaChunk record
3. MEETING ENDS
Backend finalizes → RPUSH "ProcessVideo" { meetingId }
4. MERGER WORKER (BLPOP "ProcessVideo")
Download all chunks from S3 (organized by participant)
Decrypt each chunk:
├─ Fetch wrapped CEK from Redis
├─ Unwrap with server RSA private key
└─ AES-GCM-decrypt(ciphertext, CEK, IV, authTag)
Per user: concatenate chunks → single MP4
Upload per-user video → S3 (participants/{userId}/merged.mp4)
Speaker analysis via FFmpeg silencedetect
Composite grid layout → final grid video
Upload final → S3 (final/meeting_grid_recording.mp4)
RPUSH "TranscodeVideo" { meetingId, finalPath }
5. TRANSCODER (BLPOP "TranscodeVideo")
Download final MP4
Single-pass multi-profile encode:
├─ 360p (640x360, 800kbps)
├─ 480p (854x480, 1000kbps)
├─ 720p (1280x720, 2.8Mbps)
└─ 1080p (1920x1080, 5Mbps)
Generate master.m3u8 playlist
Generate poster.jpg + sprites.jpg + thumbnails.vtt
Upload HLS artifacts → S3
Report READY → Backend
Low-Level Design (LLD)
1. Encryption Protocol
┌──────────────────────────────────────────────────────────────────┐
│ Encryption Architecture │
├──────────────────────────────────────────────────────────────────┤
│ │
│ CEK (256-bit random) ──── RSA-OAEP-256 ────► Server Public Key │
│ │ │
│ ▼ │
│ Chunk ──► AES-256-GCM (CEK, IV) ──► Ciphertext + Auth Tag │
│ │ │ │
│ │ ▼ │
│ │ Upload to S3 + Metadata │
│ │ │
│ IV = first 12 bytes of SHA-256(userId:sequenceNumber) │
│ Auth Tag = 128 bits (appended to ciphertext) │
│ │
│ Stored per chunk (MediaChunk table): │
│ - isEncrypted: true │
│ - encryptionAlgorithm: "AES-GCM" │
│ - encryptionIv: base64(iv) │
│ - encryptionTagBits: 128 │
│ │
│ Wrapped CEK stored in Redis: │
│ Key: meeting:wrapped-cek:{meetingId}:{participantId} │
│ TTL: 7 days │
└──────────────────────────────────────────────────────────────────┘
2. Database Schema (Core Models)
┌───────────────┐ ┌──────────────────┐ ┌──────────────────┐
│ User │ │ Meeting │ │ MediaChunk │
├───────────────┤ ├──────────────────┤ ├──────────────────┤
│ id (PK) │────>│ id (PK) │────>│ id (PK) │
│ email (UQ) │ │ roomId (UQ) │ │ meetingId (FK) │
│ password │ │ userId (FK) │ │ uploaderUserId │
│ googleId (UQ) │ │ passcode │ │ sequenceNumber │
│ githubToken │ │ recordingState │ │ bucketLink │
│ tier │ │ isEnded │ │ isEncrypted │
│ isVerified │ │ startedAt │ │ encryptionAlgorithm
│ │ │ endedAt │ │ encryptionIv │
└───────────────┘ └──────────────────┘ └──────────────────┘
│ │
│ │
▼ ▼
┌───────────────┐ ┌──────────────────┐ ┌──────────────────┐
│ Notification │ │FinalRecording │ │ParticipantSource │
├───────────────┤ ├──────────────────┤ ├──────────────────┤
│ id (PK) │ │ id (PK) │ │ id (PK) │
│ userId (FK) │ │ meetingId (FK,UQ)│ │ meetingId (FK) │
│ type (Enum) │ │ videoLink │ │ participantId │
│ message │ │ version │ │ videoUrl │
│ metadata (JSON)│ │ visibleToEmails[]│ │ durationMs │
│ isRead │ │ audioLink │ │ fileSizeBytes │
└───────────────┘ └──────────────────┘ └──────────────────┘
┌───────────────────────────────── Editor Domain ─────────────────────────────────┐
│ │
│ ┌────────────────┐ ┌──────────────┐ ┌────────────────┐ ┌─────────────┐│
│ │ EditorProject │───>│ EditorTrack │───>│ EditorClip │ │EditorAsset ││
│ ├────────────────┤ ├──────────────┤ ├────────────────┤ ├─────────────┤│
│ │ id (PK) │ │ id (PK) │ │ id (PK) │ │ id (PK) ││
│ │ ownerId (FK) │ │ projectId(FK)│ │ trackId (FK) │ │ projectId ││
│ │ meetingId (FK)│ │ type (Enum) │ │ sourceAssetId │ │ participantKey│
│ │ sourceMode │ │ order (UQ) │ │ timelineStartMs│ │ url │
│ │ status │ │ visible │ │ durationMs │ │ type (Enum) │
│ │ durationMs │ │ muted │ │ sourceStartMs │ │ waveformUrl │
│ │ fps/width/hgt │ │ participantKey│ │ metadata(JSON) │ └─────────────┘
│ └────────────────┘ └──────────────┘ └────────────────┘ │
│ │ │
│ ▼ │
│ ┌────────────────┐ ┌──────────────┐ ┌────────────────┐ │
│ │ EditorOverlay │ │ ExportJob │ │ MulticamLayout│ │
│ ├────────────────┤ ├──────────────┤ ├────────────────┤ │
│ │ projectId (FK) │ │ projectId │ │ projectId (FK) │ │
│ │ type (Enum) │ │ status (Enum)│ │ name │ │
│ │ content (JSON) │ │ progress │ │ viewMode (Enum)│ │
│ │ timelineStartMs│ │ outputUrl │ │ rows/cols │ │
│ │ transform(JSON)│ │ error │ │ segments[] │ │
│ │ style (JSON) │ └──────────────┘ └────────────────┘ │
│ │ animation (JSN)│ │
│ └────────────────┘ │
└──────────────────────────────────────────────────────────────────────────────────┘
Key Enums (15 total):
-
RecordingState:IDLE → RECORDING → UPLOADING → PROCESSING → READY | FAILED -
ExportStatus:QUEUED → PROCESSING → DONE | FAILED -
SourceMode:FINAL | MULTITRACK -
UserTier:FREE | PRO | ENTERPRISE | ADMIN -
ViewMode:GRID | SINGLE | PIP | CUSTOM
3. Video Editor Architecture (LLD)
┌────────────────────────────────────────────────────────────────────────────┐
│ Editor Architecture (Frontend) │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌───────────────────┐ │
│ │ useEditor │ │usePlayback │ │useTrack │ │useEditorHistory │ │
│ │ Project │ │State │ │Operations │ │(Undo/Redo, 100 │ │
│ │(Auto-save │ │(Play/Pause │ │(CRUD clips, │ │ snapshots) │ │
│ │ 1s debounce)│ │ Seek, │ │ split, │ │ │ │
│ │ │ │ Source↔Time)│ │ transitions)│ │ │ │
│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ └───────────────────┘ │
│ └────────────────┼────────────────┘ │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────────────┐ │
│ │ Editor.tsx │ │
│ │ ┌─────────────┐ ┌──────────────┐ ┌─────────────────────────┐ │ │
│ │ │ Timeline │ │ CanvasPlayer │ │ EditorPanel │ │ │
│ │ │ (Horizontal │ │ (request │ │ ┌─────────────────┐ │ │ │
│ │ │ scrollable │ │ Animation │ │ │ Tabs: Text, │ │ │ │
│ │ │ tracks) │ │ Frame loop) │ │ │ Transitions, │ │ │ │
│ │ │ │ │ │ │ │ Presets, │ │ │ │
│ │ │ Ruler │ │ <video> → │ │ │ Effects, │ │ │ │
│ │ │ Track lanes │ │ <canvas> │ │ │ Transform, │ │ │ │
│ │ │ Clip blocks │ │ compositing │ │ │ Multicam │ │ │ │
│ │ │ Transition │ │ + overlays │ │ └─────────────────┘ │ │ │
│ │ │ indicators │ └──────────────┘ └─────────────────────────┘ │ │
│ │ └─────────────┘ │ │
│ └──────────────────────────────────────────────────────────────────┘ │
│ │
│ Canvas Rendering Pipeline (useCanvasVideo + RenderLoop): │
│ requestAnimationFrame │
│ ├─ Read current video frame from hidden <video> element │
│ ├─ Apply transform (stretch, offset, trim) │
│ ├─ Render transition between clips (xfade, slide, wipe, etc.) │
│ ├─ Composite overlays (text with font/shadow/animation) │
│ └─ Draw to visible <canvas> │
└────────────────────────────────────────────────────────────────────────────┘
Canvas Rendering Pipeline:
requestAnimationFrame
├─ Read current frame from hidden <video> element
├─ Apply transform (stretch, offset, trim enforcement)
├─ Compute active transition between clips
│ └─ Use TransitionRenderer for 33+ types (fade, slide, wipe, blur, zoom, cube, etc.)
├─ Composite overlays (TEXT type with animations: fade, slide, typewriter, bounce)
└─ Draw to visible <canvas>
Editor Worker (FFmpeg) Render Pipeline:
processRenderJob()
├─ Collect render clips from project data
├─ Download source assets from S3 (content-addressable cache)
├─ For each clip:
│ ├─ Apply trim + scale
│ ├─ Apply effects (blur, color grade, chroma key, speed ramp)
│ ├─ Apply preset (shake, glitch, VHS, cinematic-bars)
│ └─ Generate ffmpeg filter graph
├─ Concatenate clips (with xfade transitions)
├─ Burn text overlays (drawtext with styling)
├─ Mix audio (volume ducking, adelay, amix)
├─ Upload to S3
├─ Update DB (ExportJob → DONE)
└─ RPUSH "TranscodeVideo"
4. Multicam Architecture (V2 Design)
Multicam is Weave's V2 feature enabling true participant-level multi-camera editing:
┌─────────────────────────────────────────────────────────────────────────┐
│ Multicam Pipeline │
│ │
│ Meeting Recording (raw chunks) │
│ │ │
│ ▼ │
│ Merger Worker: │
│ ├─ Reconstruct per-participant MP4s from chunks │
│ ├─ Upload to: participants/{userId}/merged.mp4 │
│ ├─ Speaker Analysis: │
│ │ ├─ ffmpeg silencedetect per participant │
│ │ ├─ Cross-reference energy peaks every 500ms window │
│ │ └─ Store SpeakerTimeline records │
│ └─ Also produce standard grid video (backward compatible) │
│ │
│ POST /api/v1/multicam/init │
│ ├─ Create EditorProject with sourceMode = MULTITRACK │
│ ├─ Seed N video tracks (one per participant) │
│ ├─ Create ParticipantSource, MulticamLayout, Framing, Priority │
│ └─ Return projectId │
│ │
│ Frontend Editor (Multicam mode) │
│ ├─ SpeakerLane: color-coded who-spoke-when bars │
│ ├─ Program/Cut track: user's camera switch decisions │
│ ├─ Participant tracks: per-person video tracks with hide/toggle │
│ ├─ PiP/Split/Grid layout presets │
│ ├─ Reframe handles: per-participant crop + zoom │
│ ├─ Auto-cut suggestions from speaker analysis │
│ └─ Camera Priority Panel: drag-sort priority │
│ │
│ Editor Worker (multicam render): │
│ ├─ Resolve program segments (user cuts or auto-follow speaker) │
│ ├─ For each segment: │
│ │ ├─ Single: crop+scale active speaker fullscreen │
│ │ ├─ PiP: overlay inset thumbnail on scaled main │
│ │ ├─ Split: hstack side-by-side (50/50 or 70/30) │
│ │ └─ Grid: xstack N-up grid layout │
│ ├─ Burn speaker labels (drawtext) │
│ ├─ Replace missing sources with placeholder (color + drawtext) │
│ ├─ Concatenate segments + apply overlays + mix audio │
│ └─ Promote + queue transcode │
└─────────────────────────────────────────────────────────────────────────┘
5. FFmpeg Filter Graph Examples
Single clip with effects:
ffmpeg -i input.mp4 -filter_complex "
[0:v]trim=start=10:end=20,setpts=PTS-STARTPTS,
scale=1920:1080,
eq=brightness=0.1:contrast=1.2:saturation=1.1,
gblur=sigma=2[v];
[0:a]atrim=start=10:end=20,asetpts=PTS-STARTPTS[a]"
-map [v] -map [a] output.mp4
PiP layout (multicam):
ffmpeg -i main.mp4 -i pip.mp4 -filter_complex "
[0:v]crop=iw*1:ih*1,scale=1920:1080[main];
[1:v]crop=iw*0.5:ih*0.5:iw*0.25:0,scale=480:270[pip];
[main][pip]overlay=W-w-20:H-h-20,
drawtext=text='Speaker':x=100:y=H-60:fontsize=24:fontcolor=white[out]"
-map [out] -map 0:a -map 1:a output.mp4
Grid layout (2x2):
ffmpeg -i a.mp4 -i b.mp4 -i c.mp4 -i d.mp4 -filter_complex "
[0:v]scale=960:540[0v]; [1:v]scale=960:540[1v];
[2:v]scale=960:540[2v]; [3:v]scale=960:540[3v];
[0v][1v]hstack=inputs=2[row1];
[2v][3v]hstack=inputs=2[row2];
[row1][row2]vstack=inputs=2,
amix=inputs=4"
output.mp4
6. API Endpoints (Key Routes)
Authentication
POST /api/v1/user/signup # Register
POST /api/v1/user/login # Login
POST /api/v1/user/verify-email # Verify 6-digit code
GET /api/v1/user/me # Current user
Meetings
POST /api/v1/meeting/create # Create with passcode
POST /api/v1/meeting/join/:id # Join by roomId
POST /api/v1/meeting/end/:id # Host end meeting
POST /api/v1/meeting/create/schedule # Schedule meeting
GET /api/v1/meeting/get/:id # Meeting details
Recording
POST /api/v1/upload-chunk # Upload encrypted chunk
POST /api/v1/recording/start/:id # Host start recording
POST /api/v1/recording/stop/:id # Host stop recording
GET /api/v1/recording/status/:id # Poll recording state
GET /api/v1/recording/page/:id # Full recording page
Editor
POST /api/v1/editor/projects # Create editor project
GET /api/v1/editor/projects/:id # Get project with tracks
PUT /api/v1/editor/projects/:id # Save project state
POST /api/v1/editor/projects/:id/exports # Queue export job
GET /api/v1/editor/exports/:jobId/stream # SSE export progress
Multicam
POST /api/v1/multicam/init # Create multicam project
GET /api/v1/multicam/projects/:id # Full multicam manifest
PUT /api/v1/multicam/projects/:id/framings # Update crop/zoom
PUT /api/v1/multicam/projects/:id/priorities# Camera priority
POST /api/v1/multicam/projects/:id/auto-cut # Generate cuts
Worker (internal)
POST /api/v1/worker/recording-status/:id # Worker → Backend callback
Encryption
GET /api/v1/keys/public # Server RSA public key (JWK)
POST /api/v1/keys/meeting/:id/wrapped-cek # Store wrapped CEK
Integrations
GET /api/v1/google/auth/url # Google OAuth URL
GET /api/v1/github # GitHub OAuth redirect
POST /api/v1/github/create/issue # Create GitHub issue
7. Key API Contracts
Upload Chunk (the most critical endpoint):
POST /api/v1/upload-chunk
Headers: Authorization: Bearer <jwt>
Body: multipart/form-data
- file: binary (min 48 bytes)
- meetingId: string
- participantId: string
- sequenceNumber: number
- startedAt: ISO datetime
- durationMs: number
- sourceMimeType: string
- encryptionAlgorithm?: "AES-GCM"
- encryptionIv?: string (base64)
- encryptionTagBits?: 128
Response: 201 { id, bucketLink }
Rate Limit: 600 req/min per user
Worker Recording Status (inter-service):
POST /api/v1/worker/recording-status/:meetingId
Headers: x-worker-token: <worker-service-jwt>
Body: {
status: "PROCESSING" | "READY" | "FAILED",
finalPath?: string,
version?: string
}
On READY: updates Meeting.recordingState, creates FinalRecording,
sends RECORDING_READY notification
On FAILED: sets recordingState to FAILED, sends RECORDING_FAILED notification
Design Decisions & Tradeoffs
| Decision | Rationale |
|---|---|
| Local recording in chunks | Eliminates quality loss from bad internet; enables E2E encryption |
| AES-256-GCM per chunk | Authenticated encryption prevents tampering; deterministic IV avoids nonce reuse |
| RSA-OAEP key wrapping | Only the server can unwrap CEKs; clients never see the private key |
| Redis Lists for queues | Simple, reliable, no extra dependency (compared to Bull/SQS) |
| Per-participant video reconstruction | Enables true multicam editing from raw sources instead of merged grid |
| FFmpeg for all video processing | Battle-tested, supports every filter we need, static binaries for portability |
| Single-pass multi-profile encode | Scales to 4 HLS profiles in one FFmpeg pass instead of 4 sequential encodes |
| Canvas-based editor preview | Full control over rendering pipeline; supports effects/transitions/overlays in real-time |
Conclusion
Weave demonstrates how to build a production-grade video conferencing platform with a focus on recording quality and post-production editing. The key architectural takeaways:
- Client-side encryption before upload ensures privacy even from the platform provider
- Chunk-based recording decouples recording quality from network quality
- Redis-backed job queues enable reliable async processing of CPU-intensive video tasks
- Per-participant source management enables powerful multicam editing scenarios
- FFmpeg as a service provides a flexible video processing pipeline without custom C++ code.
Weave is open source. GitHub Repository
Top comments (0)