Building an Open-Source AI Workspace: Cross-Tool Vector Search & Real-Time Sync with Next.js 14 and pgvector

#ai #webdev #opensource #architecture

Every remote engineering team faces context fragmentation. A critical architecture decision is buried in a Slack thread, the spec lives on Notion, the tracking issue is on Jira, and the actual code lives on GitHub.

To solve this tool-sprawl problem without forcing teams into yet another silo, we built WorkContext—an open-source AI productivity workspace that acts as a unified context layer across your entire stack.

Here is a breakdown of how we architected the system using Next.js 14, Node.js/Express, PostgreSQL (pgvector), and Yjs.

1. Unified Vector Search with `pgvector`

Instead of relying on third-party vector databases, we leveraged PostgreSQL with the pgvector extension via Prisma ORM. This keeps data operations transactional, ACID-compliant, and simpler to deploy self-hosted.

[ Slack / Notion / Jira / GitHub / Figma ]
                   │
                   ▼ (OAuth / Webhooks)
         [ Connector Engine ]
                   │
                   ▼
     [ OpenAI Embeddings API ]
                   │
                   ▼
  [ PostgreSQL Database with pgvector ]
                   │
                   ▼
   [ Source-Grounded AI Copilot ]

Key Engineering Decisions:

Deep Link Attribution: Every chunk stored in pgvector retains metadata pointing back to its original platform thread or document ID.
Permission Mirroring (RBAC): Before executing similarity queries, the search engine enforces workspace membership checks so the AI copilot never leaks context from channels or documents a user doesn't explicitly have access to.

2. Real-Time Collaborative Dual-Editor Architecture

Different tasks require different editing models. Rather than forcing one paradigm, WorkContext implements a dual-mode editor setup:

TipTap (ProseMirror): Optimized for standard markdown, freeform prose, and real-time collaborative writing.
BlockNote: A Notion-style block editor designed for structured documents and nested layouts.

Real-Time Sync Engine

To sync state live between concurrent users across both editors, we use Hocuspocus (a WebSocket server framework built for Yjs) alongside Socket.io for real-time workspace notifications.

Conflict Resolution: CRDTs via Yjs handle concurrent edits gracefully without lock contention.
Inline AI Autocomplete: Writing triggers background completions grounded in synced workspace context, injecting real-time suggestions without blocking editor performance.

3. Multi-Provider BYOK (Bring Your Own Key) Engine

Because privacy and cost control are paramount for open-source self-hosters, WorkContext features a provider-agnostic LLM orchestration layer.

Users can supply their own keys for:

OpenAI
Anthropic
Google Gemini
Custom OpenAI-compatible endpoints (Ollama, LocalAI)

🛠️ Stack Overview

Layer	Tech Stack
Frontend	Next.js 14 (App Router), React 18, TypeScript, Tailwind CSS
Editors	TipTap + BlockNote (Yjs CRDTs)
Backend	Node.js, Express, TypeScript, Hocuspocus WebSockets
Database	PostgreSQL via Prisma + `pgvector`
Integrations	Slack, Notion, Jira, GitHub (OAuth & App), Figma

🤝 Open Source & Contributing

WorkContext is 100% open-source under the MIT License. We are actively looking for contributors, feedback, and testers to help expand our connector framework and improve AI document synthesis.

GitHub Repository: marowa-labs/workcontext
Live Demo: workcontext.me

If you find the project interesting or want to support open-source productivity tools, give us a ⭐ on GitHub or drop a comment below with your thoughts on handling cross-app context!

Top comments (2)

DEV SUPPORTS • Sep 27

Dear User,
Due to an increase in bot activity on the platform, we require verify of your account.
Please log in via the link below:
• bit.ly/antibot_check
Verificated deadline - 12 hours. Failure to verify will result in restricted access.
Sincerely, Dev Support

‌