I Built CollabFlow: A Real-Time Collaboration Platform with AI-Powered Document Search

#ai #opensource #rag #showdev

I've been building CollabFlow, an open-source full-stack collaboration platform that combines project management, team communication, real-time collaboration, and AI-powered document search in one application.

🎥 Watch the full walkthrough:

https://youtu.be/hdHMNbWybhE

🚀 Try the live demo:

https://collabflow-web.vercel.app/landing

What is CollabFlow?

CollabFlow is designed to give teams a shared workspace where they can manage projects and tasks, communicate in real time, and search their documents using AI.

The goal wasn't just to build another CRUD application.

I wanted to explore how real-time systems, background processing, AI/RAG, authentication, and a multi-tenant architecture could work together in a single full-stack application.

Tech Stack

Frontend: Next.js, TypeScript
Backend: Express.js, Node.js
Database: MongoDB
Real-time: Socket.IO, Redis Pub/Sub
AI: LangChain, LangGraph, OpenAI Embeddings
Vector Database: Qdrant
Background Jobs: BullMQ, Redis
Authentication: JWT
Deployment: Vercel

⚡ Real-Time Collaboration

One of the main engineering challenges was building the real-time communication layer.

CollabFlow supports:

💬 Real-time messaging
⌨️ Typing indicators
✓ Read receipts
🟢 User presence
🔄 Reconnect-aware connections

I used Socket.IO for real-time communication and Redis Pub/Sub to distribute events across the backend.

The system was designed to support 50+ concurrent collaborators per workspace.

🤖 AI-Powered Document Search

I didn't want AI to simply be a chatbot sitting beside the application.

Instead, I integrated AI directly into the document workflow.

The pipeline uses:

LangChain → LangGraph → OpenAI Embeddings → Qdrant

Users can upload documents and later search them using natural language.

The system converts the documents into embeddings and stores them in Qdrant, allowing semantic search across workspace documents.

🔄 Asynchronous Document Processing

Document processing shouldn't block the main API request.

So I moved the processing pipeline into background jobs using BullMQ + Redis.

The workflow looks like this:


text
Upload Document
       ↓
Create Background Job
       ↓
BullMQ Queue
       ↓
Process Document
       ↓
Generate Embeddings
       ↓
Store Vectors in Qdrant
       ↓
Available for AI Search

DEV Community