DEV Community: Rishiraj Bal

Building a Real-Time Chat Application: A Deep Dive into Modern Web Technologies

Rishiraj Bal — Sun, 27 Jul 2025 23:03:56 +0000

Introduction

In today's interconnected world, real-time communication has become a fundamental requirement for modern web applications. Whether it's team collaboration tools, customer support systems, or social platforms, the ability to exchange messages instantly is crucial for user engagement.

Recently, I built ChatBay - a full-stack real-time chat application that demonstrates the power of combining modern web technologies. This project showcases how to create a scalable, feature-rich chat system using React, Node.js, Socket.IO, and Material-UI. This was also my first project built with cursor
and i wrote the baseline code only.

In this post, I'll walk you through the technical architecture, key implementation decisions, and the challenges I faced while building this application.

Project Overview

ChatBay is a real-time chat application that supports both group conversations and private messaging. The application features:

Real-time messaging with instant delivery
Group chat rooms with dynamic creation
Private messaging between users
Typing indicators and user presence
Role-based access control (Master, Admin, Guest)
Emoji picker with 100+ emojis
Responsive design for all devices
AI assistant integration

Technical Architecture

The Hybrid Approach

One of the most interesting aspects of this project is its hybrid architecture. Instead of choosing between REST APIs and WebSockets exclusively, I combined both approaches based on their strengths:

REST APIs for:

Room management (create, delete, list)
User authentication
CRUD operations

WebSockets for:

Real-time messaging
Typing indicators
User presence updates
Live notifications

This approach provides the best of both worlds: the reliability and simplicity of REST APIs for data operations, and the real-time capabilities of WebSockets for live communication.

Backend Implementation

The backend is built with Node.js and Express, using Socket.IO for real-time communication:

// Server setup with CORS configuration
const io = new Server(httpServer, {
  cors: {
    origin: clientUrl,
    methods: ["GET", "POST", "DELETE"],
    credentials: true
  }
});

// In-memory data storage
const activeUsers = new Map();
const rooms = new Map();
const privateChats = new Map();

Key Design Decisions:

In-Memory Storage: For simplicity and performance, I chose to store data in memory using JavaScript Maps. This eliminates database complexity while providing fast access and automatic cleanup on server restart.
Event-Driven Architecture: All real-time features are implemented using Socket.IO events, creating a loosely coupled system that's easy to extend.
Role-Based Access Control: The application implements three user types with different permission levels, demonstrating how to handle authorization in real-time applications.

Frontend Implementation

The frontend is built with React 19 and Material-UI, using modern hooks for state management:

// Socket connection with environment variables
const socket = io(import.meta.env.VITE_SOCKET_URL || 'http://localhost:3000');

// State management with React hooks
const [user, setUser] = useState(null);
const [messages, setMessages] = useState([]);
const [rooms, setRooms] = useState([]);
const [connectionStatus, setConnectionStatus] = useState('connecting');

Key Features:

Real-Time State Synchronization: Messages, user lists, and room information are synchronized across all connected clients in real-time.
Connection Status Monitoring: The application provides visual feedback about connection status, helping users understand when they're disconnected.
Responsive Design: Using Material-UI's component system ensures a consistent, professional look across all devices.

Technical Challenges and Solutions

Challenge 1: Managing Real-Time State

Problem: Keeping the UI synchronized with real-time events while maintaining good performance.

Solution: Implemented a combination of local state management with React hooks and careful event handling:

// Efficient state updates
socket.on('new_group_message', (message) => {
  if (currentRoom === message.roomName) {
    setMessages(prev => [...prev, message]);
  }
});

// Optimistic updates for better UX
const handleSendMessage = () => {
  if (newMessage.trim() && user) {
    socket.emit('group_message', {
      roomName: currentRoom,
      message: newMessage.trim(),
      userId: user.userId
    });
    setNewMessage(''); // Clear input immediately
  }
};

Challenge 2: Handling User Disconnections

Problem: Users can disconnect unexpectedly, leaving the application in an inconsistent state.

Solution: Implemented comprehensive disconnect handling:

socket.on('disconnect', () => {
  const userId = socket.userId;
  if (userId && activeUsers.has(userId)) {
    const user = activeUsers.get(userId);

    // Remove from current room
    if (user.currentRoom && rooms.has(user.currentRoom)) {
      const room = rooms.get(user.currentRoom);
      room.participants = room.participants.filter(p => p !== userId);
      socket.to(user.currentRoom).emit('user_left_room', {
        username: user.username,
        roomName: user.currentRoom
      });
    }

    // Remove from active users
    activeUsers.delete(userId);
    io.emit('user_list_updated', Array.from(activeUsers.values()));
  }
});

Challenge 3: Environment Configuration

Problem: Managing different configurations for development and production environments.

Solution: Used Vite's environment variable system with proper fallbacks:

Key Learning Outcomes

1. WebSocket Best Practices

Event Naming: Use descriptive event names that clearly indicate the action and data type
Error Handling: Always implement proper error handling for connection issues
Reconnection Logic: Provide automatic reconnection with user feedback
Data Validation: Validate all incoming data on both client and server

2. State Management Patterns

Functional Updates: Use functional updates for state that depends on previous values
Event Filtering: Only update state when the event is relevant to the current context
Optimistic Updates: Update UI immediately for better user experience

3. Security Considerations

Input Validation: Validate all user inputs on the server side
Permission Checks: Implement role-based access control for sensitive operations
CORS Configuration: Properly configure CORS for production deployments

4. Performance Optimization

Efficient Re-renders: Use React.memo and useCallback for expensive operations
Message Batching: Consider batching multiple messages for better performance
Memory Management: Clean up event listeners and timers properly

Deployment and Production Considerations

Environment Configuration

The application uses environment variables for configuration management:

Deployment Strategy

Backend: Deployed on Render with automatic environment variable configuration
Frontend: Built with Vite and served as static files
CORS: Properly configured for cross-origin requests between frontend and backend

Future Enhancements

While the current implementation provides a solid foundation, there are several areas for improvement:

Database Integration: Replace in-memory storage with a persistent database
Message Persistence: Store messages permanently for chat history
File Sharing: Add support for image and file sharing
Push Notifications: Implement browser notifications for new messages
Message Encryption: Add end-to-end encryption for private messages
Scalability: Implement horizontal scaling with Redis for session management

Conclusion

Building ChatBay was an excellent learning experience that reinforced several important concepts in modern web development:

Real-time communication is complex but manageable with the right tools
Hybrid architectures can provide the best user experience
State management in real-time applications requires careful consideration
Security and performance should be considered from the beginning

The project demonstrates how modern web technologies can be combined to create powerful, user-friendly applications. The hybrid approach of REST APIs and WebSockets provides flexibility and scalability while maintaining simplicity.

For developers looking to build similar applications, I recommend starting with a clear understanding of the requirements, choosing the right tools for each job, and implementing features incrementally. Real-time applications can be complex, but with proper planning and the right architecture, they're definitely achievable.

Resources

Live Demo: https://chatbay.onrender.com
GitHub Repository: backend
GitHub Repository:Frontend
Socket.IO Documentation: https://socket.io/docs
Material-UI Documentation: https://mui.com

This project showcases the power of modern web technologies in creating engaging, real-time user experiences. Whether you're building a chat application, a collaborative tool, or any real-time feature, the patterns and techniques demonstrated here can serve as a solid foundation.

My post on GenAI!!

Rishiraj Bal — Tue, 08 Jul 2025 23:09:49 +0000

Rishiraj Bal

Jul 8 '25

RishiGPT: Building a Multi-Agent, Memory-Aware AI with Streamlit, LangChain & Pinecone

Comments

4 min read

RishiGPT: Building a Multi-Agent, Memory-Aware AI with Streamlit, LangChain & Pinecone

Rishiraj Bal — Tue, 08 Jul 2025 23:05:15 +0000

In this post, I’ll break down every single component of RishiGPT — an AI-native, developer-first chatbot and orchestration system that combines modern LLM stacks, Retrieval-Augmented Generation (RAG), graph-based state machines, and persistent vector memory to push beyond the usual “prompt in, text out” paradigm.

I’ll cover:

The core design goals.
How the Streamlit apps are structured.
How file-based RAG works (PDF, Text, Web, GitHub).
How persistent Pinecone indexes integrate with Cohere embeddings.
How LangGraph adds conditional flows and multi-agent orchestration.
How this grows into true multimodal and voice-enabled AI.
Exact repo layout, API key strategy, and gotchas.
Full code snippets showing how each piece plugs together.

Let’s unpack it, one layer at a time.

What Problem Does RishiGPT Solve?

Most chatbots are either:

A plain LLM frontend with zero context memory.
A simple PDF Q&A app that forgets everything once you close the tab.
Or a brittle RAG pipeline stuck on one format.

RishiGPT’s mission is to unify live web search, file-based RAG, repository indexing, graph-based memory, and multi-step agent orchestration — all wrapped in a single developer-friendly interface that runs on Streamlit.

It’s not just a bot — it’s a framework for building knowledge workflows:

Embed once, query forever.
Search your own data, plus the web.
Use memory to keep context alive.
Orchestrate multi-agent tasks conditionally.
Expand into voice and image generation when funded.

Key Tech Choices

Streamlit: Fast prototyping for interactive apps.
LangChain: File loaders, chunking, embeddings, Conversational Retrieval Chain.
FAISS: Local vector DB for in-session RAG.
Pinecone: Persistent vector storage.
Cohere: Cheap, robust English embeddings.
Groq: High-performance LLaMA inference.
LangGraph[Currently in development]: Graph-based, conditional multi-step orchestration.

The Main Chatbot: `RishiGPT`

1. Web Search Agent

When you enable web search via a sidebar checkbox:

Loads the serpapi LangChain tool.
Sets up a ZERO_SHOT_REACT_DESCRIPTION agent.
Passes user queries to SerpAPI, pipes results back through Groq’s LLaMA.
Uses a ConversationBufferMemory for minimal in-session memory.

from langchain.agents import load_tools, initialize_agent, AgentType

tools = load_tools(["serpapi"], llm=model)
agent = initialize_agent(
    llm=model,
    tools=tools,
    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
    verbose=False,
    memory=None
)

2. File-Based RAG

Select:

PDF
Text
Website
GitHub repo

Each uses:

LangChain loaders: PyPDFLoader, TextLoader, WebBaseLoader, GitLoader.
RecursiveCharacterTextSplitter for chunking.
HuggingFaceEmbeddings for local embeddings.
FAISS for local vector store.

Example PDF chunking:

splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
chunks = splitter.split_documents(loader.load())

The ConversationalRetrievalChain is built on the fly:

chain = ConversationalRetrievalChain.from_llm(
    llm=model,
    retriever=db.as_retriever(search_kwargs={"k": 5}),
    memory=session_state.rag_memory,
    return_source_documents=True,
    combine_docs_chain_kwargs={"prompt": prompt_template}
)

3. GitHub Repo RAG

Clone a public repo on the fly:

from git import Repo

repo = Repo.clone_from(url, to_path=tempdir)
loader = GitLoader(repo_path=tempdir, branch=repo.head.reference)
docs = loader.load()

Same split + embed + FAISS pipeline applies.

4. Multi-Mode Toggle

The sidebar logic ensures you can’t run file RAG and web search at the same time — avoids mixing conflicting contexts.

if use_rag:
    use_serp = False  # or vice versa

5. Persistent Memory Mode

The separate Embedding Station and Deposition Chat handle permanent storage:

CohereEmbeddings → Pinecone.
Create new index → chunk docs → embed → store.
Later, use PineconeVectorStore retriever in another Streamlit app.
Keeps your knowledge forever.

LangGraph Orchestration: `RishiGPT_PLUS` [CURRENTLY IN DEVELOPMENT]

This is where things get interesting.

A simple state graph:

Pick a model: Groq, OpenAI, Cohere, Gemini.
Pick a task: Chatbot, RAG, Web Search.
If RAG: pick flavor → In-session, Pinecone, Hybrid.
End or loop.

Each node is a function:

from langgraph.graph import StateGraph, add_messages, START, END

class RishiGPTPLUS(TypedDict):
    messages: Annotated[list, add_messages]
    model: str

def choose_model(state: RishiGPTPLUS) -> str:
    return "groq"

def model_groq(state: RishiGPTPLUS) -> RishiGPTPLUS:
    return {"messages": state["messages"] + ["Using GROQ"], "model": "groq"}

The graph:

build3 = StateGraph(RishiGPTPLUS)
build3.add_node("choose_model", choose_model)
build3.add_node("groq", model_groq)
build3.add_edge(START, "choose_model")
build3.add_edge("choose_model", "groq")
...
graph3 = build3.compile()

This is a blueprint for future expansions:

Branch flows.
Conditional tasks.
Multi-agent orchestration.

How Persistent Memory Works

Create index via Embedding Station.
Use CohereEmbeddings (embed-english-light-v3.0).
Pinecone index = store chunks.
Later, Deposition Chat pulls it:

pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])
retriever = PineconeVectorStore(index=index, embedding=embeddings).as_retriever()

Future Roadmap

LangGraph orchestration: True multi-agent, dynamic workflows.

RAGAnything: PDFs, web, GitHub, but also:

Video transcripts.
Spreadsheets.
Notion wikis.
HTML dumps.
Raw JSON, XML.
Audio pipelines.
Slide decks, research datasets.

Memory-aware chat: Graph-based, chunked vector memory for full context retention.

Voice + Images: When funded, plug in DALL·E / Stable Diffusion + TTS/STT.
API usage isn’t cheap — contributors can share OpenAI keys to offset costs.

n8n Connectors: Run real APIs, webhooks, databases from chat.

Role Switching: Dev co-pilot, student tutor, research agent — switch modes instantly.

How to Run Locally

git clone https://github.com/Rishirajbal/RishiGPT.git
cd RishiGPT
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
streamlit run app_2.py

Create API_KEYS.env:

GROQ_API_KEY="..."
SERPAPI_API_KEY="..."

Or use .streamlit/secrets.toml for Streamlit Cloud.

Final Thoughts

RishiGPT is not just a chatbot — it’s a foundation for your own self-hosted LLM lab:

One repo for ad-hoc file chat.
One repo for orchestration experiments.
One repo for persistent RAG memory.

It’s easy to fork, adapt, and extend:

Swap Groq for OpenAI or Cohere.
Use LangGraph for custom flows.
Plug in Pinecone or local FAISS.
Add new loaders — CSV, HTML, Notion.
Wire up image or audio models when budget permits.

Repos

Questions?
Open an issue, fork it, push a PR.
RishiGPT is open to your ideas.

Author

Rishiraj Bal — Building practical LLM systems, robust RAG pipelines, and next-generation AI orchestration frameworks.

DEV Community: Rishiraj Bal

Building a Real-Time Chat Application: A Deep Dive into Modern Web Technologies

Introduction

Project Overview

Technical Architecture

The Hybrid Approach

Backend Implementation

Frontend Implementation

Technical Challenges and Solutions

Challenge 1: Managing Real-Time State

Challenge 2: Handling User Disconnections

Challenge 3: Environment Configuration

Key Learning Outcomes

1. WebSocket Best Practices

2. State Management Patterns

3. Security Considerations

4. Performance Optimization

Deployment and Production Considerations

Environment Configuration

Deployment Strategy

Future Enhancements

Conclusion

Resources

My post on GenAI!!

RishiGPT: Building a Multi-Agent, Memory-Aware AI with Streamlit, LangChain & Pinecone

RishiGPT: Building a Multi-Agent, Memory-Aware AI with Streamlit, LangChain & Pinecone

What Problem Does RishiGPT Solve?

Key Tech Choices

The Main Chatbot: RishiGPT

1. Web Search Agent

2. File-Based RAG

3. GitHub Repo RAG

4. Multi-Mode Toggle

5. Persistent Memory Mode

LangGraph Orchestration: RishiGPT_PLUS [CURRENTLY IN DEVELOPMENT]

How Persistent Memory Works

Future Roadmap

How to Run Locally

Final Thoughts

Repos

Author

The Main Chatbot: `RishiGPT`

LangGraph Orchestration: `RishiGPT_PLUS` [CURRENTLY IN DEVELOPMENT]