DEV Community: Vivek Yadav

Building a Production-Ready AI ChatBot Platform with Real-Time Analytics

Vivek Yadav — Mon, 17 Nov 2025 06:23:11 +0000

🎯 Introduction

In this article, I'll walk you through the architecture and implementation of a complete, production-ready AI chatbot platform that I've built from scratch. This isn't just another chatbot tutorial—it's a comprehensive system featuring OAuth 2.0 authentication, real-time WebSocket communication, OpenAI integration, and a full-featured analytics dashboard.

What makes this project special?

🏗️ Microservices Architecture: Four independent services working together seamlessly
🔒 Enterprise-Grade Security: OAuth 2.0 with JWT tokens and role-based access control
💬 Real-Time Communication: WebSocket support for instant messaging
📊 Integrated Analytics: Live metrics and user activity tracking
🚀 Production Ready: Docker deployment, comprehensive testing, and cross-platform support
🎨 Modern UI: Beautiful React interface with gradient themes and responsive design

Tech Stack:

Frontend: React 18, React Router, WebSocket API
Backend: FastAPI (Python 3.12+), SQLAlchemy, JWT
AI: OpenAI GPT-4
Database: SQLite (easily upgradeable to PostgreSQL)
Deployment: Docker, Docker Compose
Testing: Pytest, end-to-end test suite

🏗️ System Architecture

The platform consists of four microservices:

┌────────────────────────────────────────────────────────────┐
│                    React Frontend (Port 3000)               │
│  Chat Interface + Analytics Panel + Real-time Updates      │
└──────────────┬─────────────────────────────────────────────┘
               │
               ▼
    ┌──────────────────┐  ┌──────────────────┐  ┌──────────────────┐
    │  Auth Service    │  │  Chat Service    │  │Analytics Service │
    │  (Port 8001)     │  │  (Port 8000)     │  │  (Port 8002)     │
    │                  │  │                  │  │                  │
    │ • User Mgmt      │  │ • Conversations  │  │ • Metrics        │
    │ • JWT Tokens     │◄─┤ • OpenAI API     │─►│ • User Activity  │
    │ • RBAC           │  │ • WebSocket      │  │ • Token Tracking │
    │ • Auto Admin     │  │ • Analytics MW   │  │ • Admin API      │
    └──────────────────┘  └──────────────────┘  └──────────────────┘

Why Microservices?

Separation of Concerns: Each service has a single, well-defined responsibility
Scalability: Scale services independently based on load
Maintainability: Easier to understand, test, and modify individual components
Technology Flexibility: Each service can use different technologies if needed
Fault Isolation: If one service fails, others continue functioning

🔐 Authentication & Authorization

OAuth 2.0 Implementation

The auth service implements OAuth 2.0 with JWT tokens:

# auth-service/auth_server/security/auth.py
from jose import JWTError, jwt
from datetime import datetime, timedelta, timezone
from passlib.context import CryptContext

SECRET_KEY = os.getenv("AUTH_SECRET_KEY")
ALGORITHM = "HS256"
ACCESS_TOKEN_EXPIRE_MINUTES = 43200  # 30 days

pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")

def create_access_token(data: dict):
    to_encode = data.copy()
    expire = datetime.now(timezone.utc) + timedelta(minutes=ACCESS_TOKEN_EXPIRE_MINUTES)
    to_encode.update({"exp": expire})
    encoded_jwt = jwt.encode(to_encode, SECRET_KEY, algorithm=ALGORITHM)
    return encoded_jwt

def verify_password(plain_password: str, hashed_password: str) -> bool:
    return pwd_context.verify(plain_password, hashed_password)

def get_password_hash(password: str) -> str:
    return pwd_context.hash(password)

Key Features:

BCrypt Password Hashing: Secure password storage with automatic salting
JWT Tokens: Stateless authentication with 30-day expiration
Role-Based Access Control: Admin, User, Manager roles
Timezone-Aware: UTC timestamps for global compatibility

Automatic Admin Setup

One of my favorite features—the system automatically creates an admin user on first startup:

# auth-service/auth_server/main.py
@app.on_event("startup")
async def startup_event():
    """Initialize database and create admin user if not exists"""
    create_db_and_tables()

    # Auto-create admin user
    admin_username = os.getenv("ADMIN_USERNAME", "admin")
    admin_password = os.getenv("ADMIN_PASSWORD", "admin123")
    admin_email = os.getenv("ADMIN_EMAIL", "admin@example.com")

    with Session(engine) as session:
        admin_user = session.exec(
            select(User).where(User.username == admin_username)
        ).first()

        if not admin_user:
            # Create admin with both admin and user roles
            admin_user = User(
                username=admin_username,
                email=admin_email,
                hashed_password=get_password_hash(admin_password)
            )
            session.add(admin_user)
            session.commit()
            session.refresh(admin_user)

            # Assign roles
            admin_role = session.exec(select(Role).where(Role.name == "admin")).first()
            user_role = session.exec(select(Role).where(Role.name == "user")).first()

            admin_user.roles.extend([admin_role, user_role])
            session.commit()

            print(f"✅ Admin user '{admin_username}' created successfully!")

No more manual SQL commands or bootstrap scripts! The admin user is ready to use immediately after deployment.

💬 Real-Time Chat with WebSocket

WebSocket Connection Handler

The chat service uses WebSocket for real-time bidirectional communication:

# openai_web_service/websocket/chat_handler.py
@router.websocket("/ws/{conversation_id}")
async def websocket_endpoint(
    websocket: WebSocket,
    conversation_id: str,
    token: str = Query(None),
    db: Session = Depends(get_db)
):
    await websocket.accept()

    try:
        # Authenticate user from JWT token
        user_data = verify_access_token(token)
        user_id = user_data.get("user_id")

        # Verify conversation ownership
        conversation = get_conversation_by_id(db, conversation_id, user_id)
        if not conversation:
            await websocket.send_json({
                "type": "error",
                "message": "Conversation not found or access denied"
            })
            await websocket.close()
            return

        # Handle messages
        while True:
            data = await websocket.receive_json()

            if data.get("type") == "message":
                # Save user message
                user_message = create_message(
                    db, conversation_id, "user", data.get("content"), user_id
                )

                # Get AI response from OpenAI
                ai_response = await get_openai_response(
                    db, conversation_id, user_message.content
                )

                # Save AI message
                ai_message = create_message(
                    db, conversation_id, "assistant", ai_response, user_id
                )

                # Send to client
                await websocket.send_json({
                    "type": "message",
                    "role": "assistant",
                    "content": ai_response,
                    "message_id": ai_message.id,
                    "timestamp": ai_message.timestamp.isoformat()
                })

    except WebSocketDisconnect:
        print(f"Client disconnected from conversation {conversation_id}")
    except Exception as e:
        await websocket.send_json({
            "type": "error",
            "message": str(e)
        })
        await websocket.close()

Key Features:

Token Authentication: Secure WebSocket connections with JWT
Conversation Ownership: Users can only access their own conversations
Auto-reconnection: Client automatically reconnects on disconnect (max 5 attempts)
Error Handling: Graceful error messages and cleanup

OpenAI Integration

The service integrates with OpenAI's GPT-4 API:

# openai_web_service/services/openai_service.py
async def get_openai_response(db: Session, conversation_id: str, user_message: str) -> str:
    """Get response from OpenAI API with conversation context"""

    # Get conversation history
    messages = get_messages_by_conversation(db, conversation_id)

    # Build context
    conversation_history = [
        {"role": msg.role, "content": msg.content}
        for msg in messages[-10:]  # Last 10 messages for context
    ]

    # Add system message
    conversation_history.insert(0, {
        "role": "system",
        "content": "You are a helpful AI assistant."
    })

    # Call OpenAI API
    response = await openai.ChatCompletion.acreate(
        model="gpt-4",
        messages=conversation_history,
        temperature=0.7,
        max_tokens=500
    )

    return response.choices[0].message.content

Context Management: The system maintains conversation context by including the last 10 messages in each API call, ensuring coherent multi-turn conversations.

📊 Analytics Dashboard

Real-Time Metrics Tracking

The analytics service tracks comprehensive metrics:

# analytics-service/analytics/services/analytics_service.py
class AnalyticsService:
    def get_summary_metrics(self, db: Session) -> dict:
        """Get overall platform statistics"""

        now = datetime.now(timezone.utc)
        today_start = now.replace(hour=0, minute=0, second=0, microsecond=0)

        return {
            "total_users": db.query(UserActivity).distinct(UserActivity.user_id).count(),
            "active_users_today": db.query(UserActivity)
                .filter(UserActivity.timestamp >= today_start)
                .distinct(UserActivity.user_id)
                .count(),
            "total_conversations": db.query(ConversationMetrics).count(),
            "total_messages": db.query(MessageMetrics).count(),
            "total_tokens_used": db.query(func.sum(MessageMetrics.total_tokens)).scalar() or 0,
            "avg_response_time": db.query(func.avg(ApiUsage.response_time))
                .filter(ApiUsage.timestamp >= today_start)
                .scalar() or 0,
            "error_rate": self._calculate_error_rate(db, today_start)
        }

Analytics Middleware

Automatic API usage tracking without cluttering application code:

# openai_web_service/middleware/analytics_middleware.py
@app.middleware("http")
async def analytics_tracking_middleware(request: Request, call_next):
    """Track all API requests for analytics"""

    start_time = time.time()

    # Process request
    response = await call_next(request)

    # Calculate response time
    response_time = time.time() - start_time

    # Extract user info from JWT token
    user_id = None
    if authorization := request.headers.get("authorization"):
        try:
            token = authorization.split("Bearer ")[1]
            user_data = verify_access_token(token)
            user_id = user_data.get("user_id")
        except:
            pass

    # Send to analytics service (fire-and-forget)
    asyncio.create_task(
        send_analytics_event({
            "event_type": "api_request",
            "user_id": user_id,
            "endpoint": str(request.url.path),
            "method": request.method,
            "status_code": response.status_code,
            "response_time": response_time,
            "timestamp": datetime.now(timezone.utc).isoformat()
        })
    )

    return response

Fire-and-forget approach: Analytics tracking is non-blocking and doesn't impact API performance.

Frontend Analytics Panel

React component displaying live metrics:

// chat-frontend/src/components/AnalyticsPanel.js
function AnalyticsPanel({ isOpen, onClose }) {
  const [metrics, setMetrics] = useState(null);
  const [loading, setLoading] = useState(true);

  useEffect(() => {
    const fetchMetrics = async () => {
      try {
        const data = await analyticsService.getSummary();
        setMetrics(data);
      } catch (error) {
        console.error('Failed to fetch analytics:', error);
      } finally {
        setLoading(false);
      }
    };

    fetchMetrics();
    const interval = setInterval(fetchMetrics, 30000); // Refresh every 30s

    return () => clearInterval(interval);
  }, []);

  return (
    <div className={`analytics-panel ${isOpen ? 'open' : ''}`}>
      <div className="analytics-header">
        <h2>📊 Analytics Dashboard</h2>
        <button onClick={onClose}>✕</button>
      </div>

      {loading ? (
        <Spinner />
      ) : (
        <div className="metrics-grid">
          <MetricCard
            icon="👥"
            title="Total Users"
            value={metrics.total_users}
            color="#4a90e2"
          />
          <MetricCard
            icon="🔥"
            title="Active Today"
            value={metrics.active_users_today}
            color="#f5a623"
          />
          <MetricCard
            icon="💬"
            title="Conversations"
            value={metrics.total_conversations}
            color="#7ed321"
          />
          <MetricCard
            icon="📨"
            title="Messages"
            value={metrics.total_messages}
            color="#bd10e0"
          />
          <MetricCard
            icon="🎯"
            title="Tokens Used"
            value={metrics.total_tokens_used.toLocaleString()}
            color="#ff6b6b"
          />
          <MetricCard
            icon="⚡"
            title="Avg Response"
            value={`${(metrics.avg_response_time * 1000).toFixed(0)}ms`}
            color="#4ecdc4"
          />
        </div>
      )}
    </div>
  );
}

Auto-refresh: Metrics update every 30 seconds without user interaction.

🐳 Docker Deployment

Docker Compose Configuration

Complete stack deployment with one command:

# docker-compose.yml
version: '3.8'

services:
  auth-service:
    build: ./auth-service
    ports:
      - "8001:8001"
    environment:
      - AUTH_SECRET_KEY=${AUTH_SECRET_KEY}
      - ADMIN_USERNAME=admin
      - ADMIN_PASSWORD=admin123
      - ADMIN_EMAIL=admin@example.com
    networks:
      - chatbot-network

  openai_web_service:
    build: ./openai_web_service
    ports:
      - "8000:8000"
    environment:
      - OPENAI_API_KEY=${OPENAI_API_KEY}
      - AUTH_SECRET_KEY=${AUTH_SECRET_KEY}
      - AUTH_SERVICE_URL=http://auth-service:8001
      - ANALYTICS_SERVICE_URL=http://analytics-service:8002
    depends_on:
      - auth-service
      - analytics-service
    networks:
      - chatbot-network

  analytics-service:
    build: ./analytics-service
    ports:
      - "8002:8002"
    environment:
      - AUTH_SECRET_KEY=${AUTH_SECRET_KEY}
      - AUTH_SERVICE_URL=http://auth-service:8001
    depends_on:
      - auth-service
    networks:
      - chatbot-network

  chat-frontend:
    build: ./chat-frontend
    ports:
      - "3000:3000"
    environment:
      - REACT_APP_AUTH_API_URL=http://localhost:8001
      - REACT_APP_CHAT_API_URL=http://localhost:8000
      - REACT_APP_ANALYTICS_API_URL=http://localhost:8002
      - REACT_APP_WS_URL=ws://localhost:8000
    depends_on:
      - auth-service
      - openai_web_service
      - analytics-service
    networks:
      - chatbot-network

networks:
  chatbot-network:
    driver: bridge

Deploy with one command:

docker-compose up --build

🚀 Cross-Platform Local Development

Platform-Specific Scripts

I created comprehensive script suites for Windows, macOS, and Linux:

Windows (scripts/windows/):

:: start-all-services.bat
@echo off
cd /d %~dp0..\..

:: Start auth service
start "Auth Service" cmd /k "cd auth-service && venv\Scripts\activate && uvicorn auth_server.main:app --reload --port 8001"

:: Start chat service
start "Chat Service" cmd /k "cd openai_web_service && venv\Scripts\activate && uvicorn main:app --reload --port 8000"

:: Start analytics service
start "Analytics Service" cmd /k "cd analytics-service && venv\Scripts\activate && uvicorn main:app --reload --port 8002"

:: Start frontend
start "Frontend" cmd /k "cd chat-frontend && npm start"

echo All services started!

Linux/Mac (scripts/linux-mac/):

#!/bin/bash
# start-all-services.sh

PROJECT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"

# Detect terminal emulator
if [[ "$OSTYPE" == "darwin"* ]]; then
    TERMINAL="Terminal"
else
    TERMINAL="gnome-terminal"
fi

# Start auth service
$TERMINAL -- bash -c "cd $PROJECT_ROOT/auth-service && source venv/bin/activate && uvicorn auth_server.main:app --reload --port 8001; exec bash"

# Start chat service
$TERMINAL -- bash -c "cd $PROJECT_ROOT/openai_web_service && source venv/bin/activate && uvicorn main:app --reload --port 8000; exec bash"

# Start analytics service
$TERMINAL -- bash -c "cd $PROJECT_ROOT/analytics-service && source venv/bin/activate && uvicorn main:app --reload --port 8002; exec bash"

# Start frontend
$TERMINAL -- bash -c "cd $PROJECT_ROOT/chat-frontend && npm start; exec bash"

echo "All services started!"

One-command setup:

# Windows
scripts\windows\setup-venv.bat
scripts\windows\start-all-services.bat

# Linux/Mac
scripts/linux-mac/setup-venv.sh
scripts/linux-mac/start-all-services.sh

🧪 Comprehensive Testing

End-to-End Test Suite

Over 50 test cases covering all functionality:

# tests/test_4_end_to_end.py
import pytest
from datetime import datetime, timezone

class TestEndToEndFlow:
    """Complete user journey tests"""

    def test_complete_user_journey_rest_api(self):
        """Test complete user flow using REST API"""

        # 1. Register user
        register_data = {
            "username": f"user_{int(datetime.now(timezone.utc).timestamp())}",
            "email": f"user_{int(datetime.now(timezone.utc).timestamp())}@test.com",
            "password": "testpass123",
            "full_name": "Test User"
        }
        register_response = requests.post(
            f"{AUTH_BASE_URL}/users/", 
            json=register_data
        )
        assert register_response.status_code == 200

        # 2. Login
        login_data = {
            "username": register_data["username"],
            "password": register_data["password"]
        }
        token_response = requests.post(
            f"{AUTH_BASE_URL}/auth/token",
            data=login_data
        )
        assert token_response.status_code == 200
        token = token_response.json()["access_token"]

        headers = {"Authorization": f"Bearer {token}"}

        # 3. Create conversation
        conv_response = requests.post(
            f"{CHAT_BASE_URL}/conversations/",
            json={"title": "Test Conversation"},
            headers=headers
        )
        assert conv_response.status_code == 200
        conversation_id = conv_response.json()["id"]

        # 4. Send message
        message_response = requests.post(
            f"{CHAT_BASE_URL}/conversations/{conversation_id}/messages",
            json={
                "conversation_id": conversation_id,
                "role": "user",
                "content": "Hello, AI!"
            },
            headers=headers
        )
        assert message_response.status_code == 200

        # 5. Get message history
        history_response = requests.get(
            f"{CHAT_BASE_URL}/conversations/{conversation_id}/messages",
            headers=headers
        )
        assert history_response.status_code == 200
        assert len(history_response.json()) >= 2  # User + AI response

        # 6. Delete conversation
        delete_response = requests.delete(
            f"{CHAT_BASE_URL}/conversations/{conversation_id}",
            headers=headers
        )
        assert delete_response.status_code == 200

    def test_websocket_real_time_messaging(self):
        """Test WebSocket real-time communication"""

        # Register and login
        token = self.get_test_token()

        # Connect WebSocket
        ws_url = f"{WS_BASE_URL}/ws/test-conversation?token={token}"
        ws = websocket.create_connection(ws_url)

        # Send message
        ws.send(json.dumps({
            "type": "message",
            "content": "Test WebSocket message"
        }))

        # Receive AI response
        response = json.loads(ws.recv())
        assert response["type"] == "message"
        assert response["role"] == "assistant"
        assert len(response["content"]) > 0

        ws.close()

Run tests:

pytest tests/ -v --cov=. --cov-report=html

🎨 UI/UX Highlights

Responsive Design

The frontend adapts seamlessly to all screen sizes:

/* chat-frontend/src/components/ChatPage.css */
.chat-page {
  display: flex;
  height: 100vh;
}

.sidebar {
  width: 300px;
  min-width: 300px;
  background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
}

.main-content {
  flex: 1;
  display: flex;
  flex-direction: column;
}

/* Mobile responsive */
@media (max-width: 768px) {
  .sidebar {
    position: fixed;
    left: -300px;
    transition: left 0.3s ease;
    z-index: 1000;
  }

  .sidebar.open {
    left: 0;
  }

  .analytics-panel {
    width: 100% !important;
  }
}

Gradient Themes

Beautiful gradient color schemes throughout:

/* Conversation list gradients */
.conversation-item {
  background: linear-gradient(135deg, rgba(255,255,255,0.1) 0%, rgba(255,255,255,0.05) 100%);
  backdrop-filter: blur(10px);
  border: 1px solid rgba(255,255,255,0.2);
}

/* Message bubbles */
.user-message {
  background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
  color: white;
}

.ai-message {
  background: linear-gradient(135deg, #f093fb 0%, #f5576c 100%);
  color: white;
}

/* Analytics metrics */
.metric-card {
  background: linear-gradient(135deg, rgba(255,255,255,0.1) 0%, rgba(255,255,255,0.05) 100%);
  backdrop-filter: blur(10px);
  box-shadow: 0 8px 32px 0 rgba(31, 38, 135, 0.37);
}

🔧 Configuration Management

3-Tier Environment System

Root .env (shared secrets):

AUTH_SECRET_KEY=your-super-secret-key-here
OPENAI_API_KEY=sk-your-openai-api-key

Service .env files (service-specific):

# auth-service/.env
PORT=8001
HOST=0.0.0.0
CORS_ORIGINS=http://localhost:3000
ADMIN_USERNAME=admin
ADMIN_PASSWORD=admin123
ADMIN_EMAIL=admin@example.com

# openai_web_service/.env
PORT=8000
HOST=0.0.0.0
AUTH_SERVICE_URL=http://localhost:8001
ANALYTICS_SERVICE_URL=http://localhost:8002
CORS_ORIGINS=http://localhost:3000

# analytics-service/.env
PORT=8002
HOST=0.0.0.0
AUTH_SERVICE_URL=http://localhost:8001
CORS_ORIGINS=http://localhost:3000

Frontend .env:

REACT_APP_AUTH_API_URL=http://localhost:8001
REACT_APP_CHAT_API_URL=http://localhost:8000
REACT_APP_ANALYTICS_API_URL=http://localhost:8002
REACT_APP_WS_URL=ws://localhost:8000

📈 Performance Optimizations

Database Optimization

Absolute Paths: Databases use absolute paths to prevent multiple database files:

# openai_web_service/engine/database.py
from pathlib import Path

# Get service directory
service_dir = Path(__file__).parent.parent

# Create data directory
data_dir = service_dir / "data"
data_dir.mkdir(parents=True, exist_ok=True)

# Database path
db_path = data_dir / "chatbot.db"
DATABASE_URL = f"sqlite:///{db_path}"

Caching Strategy

Static Asset Caching in Nginx configuration:

# chat-frontend/nginx.conf
server {
    listen 3000;
    location / {
        root /usr/share/nginx/html;
        index index.html;
        try_files $uri $uri/ /index.html;
    }

    # Cache static assets
    location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg)$ {
        expires 1y;
        add_header Cache-Control "public, immutable";
    }
}

WebSocket Connection Pooling

Efficient connection management:

// chat-frontend/src/hooks/useChat.js
const [ws, setWs] = useState(null);
const reconnectAttemptsRef = useRef(0);
const maxReconnectAttempts = 5;

const connect = useCallback(() => {
  if (ws?.readyState === WebSocket.OPEN) {
    ws.close();
  }

  const token = localStorage.getItem('token');
  const websocket = new WebSocket(
    `${WS_URL}/ws/${conversationId}?token=${token}`
  );

  websocket.onopen = () => {
    console.log('WebSocket connected');
    reconnectAttemptsRef.current = 0;
  };

  websocket.onclose = () => {
    if (reconnectAttemptsRef.current < maxReconnectAttempts) {
      setTimeout(() => {
        reconnectAttemptsRef.current++;
        connect();
      }, 2000 * reconnectAttemptsRef.current);
    }
  };

  setWs(websocket);
}, [conversationId]);

🛡️ Security Best Practices

1. Password Security

BCrypt hashing with automatic salting
Minimum password complexity requirements
Password updates require current password

2. JWT Token Security

HS256 algorithm
30-day expiration (configurable)
Token refresh mechanism
Secure storage in localStorage (HTTPS only in production)

3. CORS Configuration

Whitelist specific origins
No wildcard (*) in production
Credentials support enabled

4. SQL Injection Prevention

SQLAlchemy ORM (parameterized queries)
Input validation with Pydantic schemas
No raw SQL queries

5. XSS Protection

React's built-in XSS protection
Content Security Policy headers
Sanitized user inputs

6. Rate Limiting (Future Enhancement)

# Future implementation
from slowapi import Limiter
from slowapi.util import get_remote_address

limiter = Limiter(key_func=get_remote_address)

@app.get("/api/endpoint")
@limiter.limit("5/minute")
async def endpoint():
    pass

📊 Monitoring & Observability

Health Check Endpoints

All services expose health checks:

@app.get("/health")
async def health_check():
    return {
        "status": "healthy",
        "service": "auth-service",
        "version": "1.1.0",
        "timestamp": datetime.now(timezone.utc).isoformat()
    }

Logging Configuration

Structured logging with levels:

import logging

logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    handlers=[
        logging.FileHandler('app.log'),
        logging.StreamHandler()
    ]
)

logger = logging.getLogger(__name__)

Metrics Collection

Analytics service tracks:

API response times
Error rates
Token usage
User activity
Conversation metrics

🚢 Deployment Strategies

Docker Production Deployment

# Build images
docker-compose build

# Deploy in detached mode
docker-compose up -d

# View logs
docker-compose logs -f

# Scale services
docker-compose up -d --scale openai_web_service=3

Environment-Specific Configurations

# docker-compose.prod.yml
services:
  openai_web_service:
    environment:
      - ENVIRONMENT=production
      - DEBUG=false
      - LOG_LEVEL=WARNING
    deploy:
      replicas: 3
      resources:
        limits:
          cpus: '0.50'
          memory: 512M

CI/CD Pipeline (Example)

# .github/workflows/deploy.yml
name: Deploy

on:
  push:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Run tests
        run: |
          pip install -r requirements.txt
          pytest tests/ -v

  deploy:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Deploy to production
        run: |
          docker-compose -f docker-compose.prod.yml up -d --build

📚 Lessons Learned

1. Start with Microservices from Day One

Breaking the system into services early made scaling and maintenance much easier. Each service can be deployed, tested, and scaled independently.

2. Invest in Developer Experience

Creating platform-specific scripts saved countless hours. One-command setup and deployment significantly lowered the barrier to entry for new contributors.

3. Security is Not Optional

Implementing OAuth 2.0, RBAC, and proper password hashing from the beginning prevented major refactoring later. Security should be baked in, not bolted on.

4. Real-Time Features Require Careful Planning

WebSocket implementation required thinking about connection management, reconnection logic, and error handling upfront. The fire-and-forget analytics tracking pattern kept the main application performant.

5. Documentation is Code

Comprehensive READMEs and setup guides made onboarding new developers seamless. The time invested in documentation paid dividends in reduced support questions.

6. Test Everything

The end-to-end test suite caught numerous edge cases and integration issues. Having 50+ tests gave confidence to refactor and add features.

🔮 Future Enhancements

Short-Term (Next 3 Months)

[ ] PostgreSQL migration for production
[ ] Redis for session management and caching
[ ] Rate limiting on API endpoints
[ ] Conversation search functionality
[ ] Message editing and deletion
[ ] File upload support
[ ] Voice input/output

Medium-Term (3-6 Months)

[ ] Multi-model support (GPT-4, Claude, Gemini)
[ ] Conversation sharing and collaboration
[ ] Custom AI assistant personas
[ ] API key management for users
[ ] Advanced analytics (charts, graphs, exports)
[ ] Mobile app (React Native)
[ ] Email notifications

Long-Term (6-12 Months)

[ ] Multi-tenancy support
[ ] Plugin system for extensions
[ ] Kubernetes deployment
[ ] GraphQL API
[ ] Real-time collaboration features
[ ] Advanced AI features (RAG, function calling)
[ ] Marketplace for custom assistants

🎓 Key Takeaways

If you're building a similar system, here are my recommendations:

Architecture: Start with a clear separation of concerns. Microservices may seem overkill initially, but they pay off quickly.
Security: Don't compromise on security. OAuth 2.0, JWT, RBAC, and proper password hashing are table stakes.
Real-Time: WebSocket adds complexity but provides a superior user experience. Plan for connection management and error recovery.
Testing: Write tests as you go. End-to-end tests are especially valuable for catching integration issues.
Documentation: Document everything. Future you (and your team) will thank you.
Developer Experience: Invest in tooling and scripts. One-command setup and deployment are worth the effort.
Observability: Build in logging, metrics, and health checks from day one. You can't debug what you can't see.

📦 Project Repository

GitHub: github.com/vcse59/ConvoAI

Quick Start

# Clone repository
git clone https://github.com/vcse59/ConvoAI.git
cd ConvoAI

# Create .env file
echo "AUTH_SECRET_KEY=your-secret-key" > .env
echo "OPENAI_API_KEY=sk-your-key" >> .env

# Docker deployment
docker-compose up --build

# Or local development (Windows)
scripts\windows\setup-venv.bat
scripts\windows\start-all-services.bat

# Or local development (Linux/Mac)
scripts/linux-mac/setup-venv.sh
scripts/linux-mac/start-all-services.sh

# Access at http://localhost:3000

Project Stats

Lines of Code: ~15,000+
Services: 4 (Frontend, Auth, Chat, Analytics)
Test Coverage: 80%+
Documentation: 6,000+ lines
Scripts: 22 platform-specific automation scripts

🤝 Contributing

Contributions are welcome! Areas where help is needed:

Additional language models integration
Mobile app development
Performance optimizations
UI/UX improvements
Documentation translations
Additional test coverage

📄 License

MIT License - See LICENSE file for details.

🙏 Acknowledgments

FastAPI: Amazing Python web framework
React: Excellent frontend library
OpenAI: GPT-4 API for AI responses
SQLAlchemy: Powerful ORM
Docker: Simplified deployment

📞 Contact

GitHub: @vcse59
Email: v.cse59@gmail.com
LinkedIn: Connect with me

Built with ❤️ using React, FastAPI, and OpenAI

If you found this article helpful, please give it a ❤️ and follow for more content on full-stack development, AI integration, and microservices architecture!

💡 Questions?

Drop your questions in the comments below! I'm happy to discuss:

Architecture decisions and tradeoffs
Implementation details
Deployment strategies
Scaling considerations
Security best practices
Testing approaches

Let's build amazing things together! 🚀

Python Learning : Serialization and De-serialization

Vivek Yadav — Thu, 28 Aug 2025 07:02:04 +0000

When building real-world applications—especially those that deal with data exchange, persistence, or communication between systems—you’ll often need a way to convert Python objects into a storable or transferable format, and then reconstruct them back when needed. This is where serialization and de-serialization come in.

1. What is Serialization?

Serialization is the process of converting a Python object (e.g., dictionary, list, class instance) into a format that can be easily stored on disk, transmitted across a network, or shared between different environments.

Python Object → Serialized Format (string/bytes)

2. What is De-serialization?

De-serialization is the reverse process—converting the serialized format back into a Python object.

Serialized Format → Python Object

3. Serialization Formats in Python

Format	Module / Library	Usage
Pickle	pickle	Native Python serialization (supports almost any Python object, but not human-readable).
JSON	json	Language-independent, human-readable, widely used for APIs and configuration files.
YAML	PyYAML (third-party)	Human-friendly, used in configs (e.g., Kubernetes, Ansible).
MessagePack / Protobuf / Avro	Third-party	libraries More compact and efficient for distributed systems.

4. Serialization with pickle

import pickle

# Sample Python object
data = {
    "name": "Alice",
    "age": 30,
    "skills": ["Python", "ML", "Data Science"]
}

# Serialization
with open("data.pkl", "wb") as file:
    pickle.dump(data, file)

# De-serialization
with open("data.pkl", "rb") as file:
    loaded_data = pickle.load(file)

print(loaded_data)

Output:

{'name': 'Alice', 'age': 30, 'skills': ['Python', 'ML', 'Data Science']}

5. Serialization with json

import json

data = {
    "name": "Bob",
    "age": 25,
    "skills": ["JavaScript", "React", "Node.js"]
}

# Serialization
json_str = json.dumps(data)   # object → string
print(json_str)

# Save to file
with open("data.json", "w") as file:
    json.dump(data, file)

# De-serialization
with open("data.json", "r") as file:
    loaded_data = json.load(file)

print(loaded_data)

Output:

{"name": "Bob", "age": 25, "skills": ["JavaScript", "React", "Node.js"]}
{'name': 'Bob', 'age': 25, 'skills': ['JavaScript', 'React', 'Node.js']}

6. Serializing Custom Objects

Using pickle
import pickle

class Person:
    def __init__(self, name, age):
        self.name = name
        self.age = age

p = Person("Eve", 28)

# Serialize
with open("person.pkl", "wb") as f:
    pickle.dump(p, f)

# Deserialize
with open("person.pkl", "rb") as f:
    p2 = pickle.load(f)

print(p2.name, p2.age)

Output:

Eve 28

Using json with custom encoder/decoder

import json

class Person:
    def __init__(self, name, age):
        self.name = name
        self.age = age

def encode_person(obj):
    if isinstance(obj, Person):
        return {"name": obj.name, "age": obj.age, "__person__": True}
    raise TypeError("Object not serializable")

def decode_person(dct):
    if "__person__" in dct:
        return Person(dct["name"], dct["age"])
    return dct

# Serialize
p = Person("Alice", 30)
json_str = json.dumps(p, default=encode_person)
print(json_str)

# Deserialize
p2 = json.loads(json_str, object_hook=decode_person)
print(p2.name, p2.age)

Output:

{"name": "Alice", "age": 30, "__person__": true}
Alice 30

7. Advanced Serialization Formats

Using MessagePack
import msgpack

data = {"id": 1, "msg": "Hello"}

# Serialize
packed = msgpack.packb(data)
print(packed)

# Deserialize
unpacked = msgpack.unpackb(packed)
print(unpacked)

Output (packed is in bytes):

b'\x82\xa2id\x01\xa3msg\xa5Hello'
{'id': 1, 'msg': 'Hello'}

8. Best Practices

1. Use json when:

Interacting with external systems.
Need human-readable formats.
Working with web APIs.

2. Use pickle only for:

Internal Python applications.
Short-term persistence.
Avoid loading untrusted pickle data.

3. Use binary formats (MessagePack/Protobuf) when:

Performance and efficiency matter.
Working with distributed systems.

9. Real-World Applications

Machine Learning: Saving/loading trained models (joblib/pickle).
Web Development: JSON payloads via REST APIs.
Configuration Management: YAML/JSON for configs.
Caching: Storing serialized objects in Redis.
Data Pipelines: Kafka/Avro for event streaming.

10. Conclusion

Serialization and de-serialization are fundamental concepts enabling persistence, sharing, and transfer of data.

Pickle: Powerful but Python-specific.
JSON: Universal and human-readable.
MessagePack/Protobuf/Avro: Efficient for distributed and large-scale apps.

Choosing the right format makes your application efficient, secure, and interoperable.

Python learning : Using Context Managers in Python

Vivek Yadav — Thu, 28 Aug 2025 06:41:15 +0000

When writing Python programs, you often need to manage resources such as files, database connections, network sockets, or locks. Properly acquiring and releasing these resources is crucial to avoid leaks, errors, or unpredictable behavior. This is where context managers come into play.

Python provides a clean and elegant way to manage resources using the with statement and context managers. In this article, we’ll dive deep into what context managers are, how they work, and how you can create your own.

What is a Context Manager?

A context manager in Python is an object that defines a runtime context to be established when executing a block of code. It ensures that certain setup and teardown actions are taken automatically, without the programmer explicitly writing them each time.

In simpler terms:

It takes care of acquiring a resource before the block starts.
It takes care of releasing/cleaning up the resource after the block ends (even if exceptions occur).

The most common way you’ve already seen context managers is when working with files:

with open("example.txt", "r") as file:
    content = file.read()
    print(content)

Here, Python:

Opens the file (enter method).
Lets you work with it inside the with block.
Closes the file automatically when the block ends, even if an exception is raised.

How Does a Context Manager Work?

A context manager uses two special methods:

__enter__(self) → Called at the beginning of the with block. It returns the resource you’ll use.
__exit__(self, exc_type, exc_value, traceback) → Called at the end of the with block. It performs cleanup. The parameters help handle exceptions inside the block.

Example: Writing a Simple Context Manager

class MyContext:
    def __enter__(self):
        print("Entering context: resource acquired")
        return "Resource"

    def __exit__(self, exc_type, exc_value, traceback):
        print("Exiting context: resource released")
        if exc_type:
            print(f"An exception occurred: {exc_value}")
        return False  # Propagate exception if any

Using it:

with MyContext() as resource:
    print("Inside with block:", resource)
    # Uncomment the next line to see exception handling
    # raise ValueError("Something went wrong")

Output:

Entering context: resource acquired
Inside with block: Resource
Exiting context: resource released

If an exception occurs, __exit__ still runs, ensuring cleanup.

Using `contextlib`for Simpler Context Managers

Python’s contextlibmodule provides utilities to make writing context managers easier.

Using contextlib.contextmanager Decorator

Instead of writing a full class with __enter__ and __exit__, you can use a generator function:

from contextlib import contextmanager

@contextmanager
def managed_resource():
    print("Acquiring resource")
    yield "Resource"
    print("Releasing resource")

Usage:

with managed_resource() as res:
    print("Using:", res)

Output:

Acquiring resource
Using: Resource
Releasing resource

This is concise and often preferred for simple use cases.

Practical Examples of Context Managers

1. File Handling

with open("data.txt", "w") as f:
    f.write("Hello, Context Managers!")
# File automatically closed

2. Database Connections

import sqlite3

with sqlite3.connect("example.db") as conn:
    cursor = conn.cursor()
    cursor.execute("CREATE TABLE IF NOT EXISTS users (id INTEGER, name TEXT)")
    cursor.execute("INSERT INTO users VALUES (1, 'Alice')")
# Connection automatically committed/closed

3. Threading Locks

import threading

lock = threading.Lock()

with lock:
    # Critical section
    print("Thread-safe operation")
# Lock automatically released

4. Suppressing Exceptions

from contextlib import suppress

with suppress(FileNotFoundError):
    with open("missing.txt") as f:
        data = f.read()
# No error raised if file is missing

5. Redirecting Output

import sys
from contextlib import redirect_stdout

with open("output.txt", "w") as f:
    with redirect_stdout(f):
        print("This goes to the file, not the console")

Best Practices with Context Managers

Use withwherever possible → It makes your code cleaner, safer, and more Pythonic.
Don’t reinvent the wheel → Check contextlibbefore writing custom managers.
Handle exceptions properly → Decide whether to suppress or propagate exceptions in __exit__.
Keep them focused → A context manager should manage a single well-defined resource.

Summary

Context managers help manage resources automatically by combining setup and teardown logic.
They rely on __enter__ and __exit__ methods.
The with statement ensures cleanup happens even if exceptions occur.
The contextlib module makes writing context managers easier.
Use them for files, network connections, database transactions, locks, logging, and more.

🔄 Google Agent-to-Agent (A2A) Communication SDK – In-Depth Guide with Python Example

Vivek Yadav — Wed, 25 Jun 2025 05:04:26 +0000

🚀 Overview

Google's Agent-to-Agent (A2A) SDK is a powerful communication layer enabling multiple conversational agents (LLMs, rule-based bots, or services) to collaborate seamlessly within a shared context. It facilitates:

Multi-agent orchestration
Shared memory and context
Agent coordination via events
Plug-and-play interoperability across modalities

Used internally at Google (e.g., Gemini, Bard integrations), the A2A SDK is now open-source and aims to become the backbone of multi-agent conversational systems.

🧠 Key Concepts

Agent
An autonomous unit capable of receiving input, taking action, and producing output.
Exchange
A communication channel that routes messages and events between agents.
Session
A persistent interaction space where multiple agents operate and share state.
Events
Messages or signals that trigger agent actions (e.g., TextInputEvent, AgentResponseEvent, ToolUseEvent).

🧱 Architecture Diagram

Example

Checkout the end-to-end implementation of A2A client and server here at repo.

🤖 Building More Advanced Agents

A2A supports event-driven composition. You can register agents that:

Call tools or external APIs
Chain multiple agents (like planners and executors)
Share session state
Handle multi-turn conversations

🌐 Use Case Examples

Use Case	Description
Conversational RAG	Planner agent routes questions, retriever agent fetches from vector DB.
Voicebot + LLM	Voice agent transcribes speech, LLM responds, TTS agent speaks back.
Customer Support	One agent fetches customer info, another handles FAQs, another resolves.
Copilot Tooling	Planner agent routes requests to different developer tools or copilots.

📌 Summary

Component	Role
Agent	Receives events, produces responses
Exchange	Event router and session manager
Event	Input/output signal used for agent communication
Session	Shared memory space across agents in a flow

Google's A2A SDK is modular, async, and developer-friendly, making it perfect for building next-gen AI agents.

Understand MCP (Model Context Protocol) with example

Vivek Yadav — Wed, 25 Jun 2025 04:50:00 +0000

Modern AI systems are shifting from static chatbots to interactive, tool-augmented agents. The Model Context Protocol (MCP) is an open protocol designed to standardize how models discover and use external tools securely and dynamically. Developed collaboratively by OpenAI, Anthropic, Google, and others, MCP enables large language models (LLMs) to interact with real-world systems like databases, APIs, and files—while maintaining modularity and security.

🧠 What is MCP?

MCP (Model Context Protocol) allows AI agents to call tools exposed via a structured interface (like HTTP + JSON-RPC) without needing to know their implementation details. An AI model can:

Discover available tools
Call tools dynamically
Use the tool output to inform its next steps
Do all this through a secure, well-defined interface

🧱 MCP Architecture Overview

📦 Key Components

Component	Role
MCP Server	Hosts tools (file read, database query, etc.) that conform to the MCP spec
MCP Client	Translates model requests into MCP tool calls
LLM (e.g. Ollama llama3.2)	The language model that plans and decides what tools to call
Chat UI	Human-facing frontend to drive conversation and display model output

🧪 Sample Use Case: AI Assistant with Arithmetic Tools

Code is available in repo

Chat with an LLM assistant
The assistant can perform arithmetic operations
This is all done via tool invocation through MCP

🔚 Conclusion

MCP is revolutionizing how LLMs interact with the outside world. It enables modular, secure, and dynamic AI experiences. With just a few lines of code, you can build AI systems that see, act, and respond based on real-time tools.

Building a Retrieval-Augmented Generation (RAG) API and Frontend with FastAPI and React Native

Vivek Yadav — Fri, 21 Mar 2025 05:47:33 +0000

Introduction

In this guide, we'll build a Retrieval-Augmented Generation (RAG) system using FastAPI for the backend and React Native for the frontend. The RAG system will allow users to interact with PDF documents by querying relevant information and generating responses using an advanced language model powered by Ollama. This system will also provide citations for the retrieved data, linking it back to the original documents.

Overview of the Architecture:

1. Backend (FastAPI):

Handles PDF uploads from the user, processes the content (text extraction), and stores the data in a vector database.
Generates embeddings using Ollama API and stores them in ChromaDB for fast retrieval.
Provides an API endpoint to process user queries by querying the vector database and returning the relevant document chunks.
Uses Ollama to generate responses to user queries based on the context retrieved from the documents.
Returns the response along with citation links for transparency and verification.

2. Frontend (React Native):

Provides a user-friendly interface to interact with the backend API.
Users can upload PDFs, view responses, and interact with the system by sending queries.
Displays responses from the backend with citations, enabling users to verify the sources of the information.
Handles user input, sends queries to the backend, and displays the resulting responses.

Backend Setup: FastAPI for PDF Processing and Query Handling

The backend is built with FastAPI, which is known for its speed and efficiency. The FastAPI server will handle PDF uploads, process text using the Ollama API, and store the results in a vector database for fast retrieval.

Backend Code (FastAPI)

1. PDF Upload and Text Extraction: The backend provides an endpoint to upload PDF files. The file is saved on the server, and its text content is extracted. The text is then chunked into smaller pieces (e.g., 500 words per chunk), making it easier to work with.

Text Extraction from PDF:

def extract_text_from_pdf(file_path):
    text = ""
    with open(file_path, "rb") as file:
        reader = pypdf.PdfReader(file)
        for page in reader.pages:
            text += page.extract_text() + "\n"
    return text

Here, we use the PyPDF library to read and extract text from the PDF. The extract_text_from_pdf function opens the file, reads each page, and appends the extracted text to a string.

2. Chunking Text for Embedding: After extracting the text, it is chunked into smaller pieces to improve embedding generation and querying performance. The chunking ensures that each document is broken down into manageable parts.

Chunking Function:

def chunk_text(text, chunk_size=500):
    words = text.split()
    return [" ".join(words[i:i+chunk_size]) for i in range(0, len(words), chunk_size)]

3. Generating Embeddings: For each chunk of text, we generate an embedding using the Ollama API. Embeddings are numerical representations of text that capture semantic meaning, allowing us to efficiently retrieve relevant document chunks when processing user queries.

Embedding Generation:

async def generate_embedding(text):
    try:
        response = httpx.post(f"{OLLAMA_API_URL}/api/embeddings", json={"model": "nomic-embed-text", "prompt": text})
        response_data = response.json()
        return response_data['embedding']
    except Exception as e:
        print(f"Error generating embedding: {e}")
        return None

4. Storing Embeddings in ChromaDB: The embeddings are stored in ChromaDB, a vector database that allows for efficient retrieval of document chunks based on their embeddings. When the user queries the system, we can compare the query's embedding with the stored embeddings to find the most relevant document chunks.

Storing Chunks in ChromaDB:

collection.add(
    ids=[f"{file.filename}-{idx}"], 
    embeddings=embedding, 
    metadatas=[{"text": chunk, "source": file.filename}], 
    documents=[chunk]
)

5. Retrieving Relevant Chunks: When the user sends a query, we generate an embedding for the query and retrieve the most relevant chunks of text by comparing the query embedding with the stored embeddings in the vector database.

Query Retrieval:

async def retrieve_relevant_chunks(query, top_k=3):
    query_embedding = await generate_embedding(query)
    results = collection.query(query_embeddings=query_embedding, n_results=top_k)
    relevant_chunks = []
    citation_links = []
    for item in results["metadatas"][0]:
        chunk_text = item["text"]
        source_file = item["source"]
        citation_url = f"http://localhost:8000/pdf/{source_file}"
        relevant_chunks.append(chunk_text)
        citation_links.append({"url" : citation_url, "title" : source_file})
    return relevant_chunks, citation_links

6. Querying Ollama for a Response: Once the relevant chunks are retrieved, we send a prompt to Ollama's language model with the context from the documents and the user’s query. Ollama generates a response, which is then sent back to the frontend.

Query Ollama:

async def query_ollama_with_context(context_chunks, model_name, user_query):
    context = "\n\n".join(context_chunks)
    prompt = f"Context: {context}\n\nUser Query: {user_query}\n\nAnswer:"
    async with httpx.AsyncClient(timeout=900) as client:
        response = await client.post(
            f"{OLLAMA_API_URL}/api/generate",
            json={"model": model_name, "prompt": prompt, "stream": False}
        )
        response_data = response.json()
        return response_data["response"]

7. Serving PDFs: The FastAPI backend also allows users to download the original PDF files via a simple API endpoint.

Serve PDFs:

@app.get("/pdf/{filename}")
async def serve_pdf(filename: str):
    file_path = os.path.join(PDF_STORAGE_DIR, filename)
    if os.path.exists(file_path):
        return FileResponse(file_path, media_type="application/pdf")
    return {"error": "File not found"}

Frontend Setup: React Native for User Interaction

The React Native frontend provides an interface for the user to interact with the system, including uploading PDFs, querying the system, and receiving responses.

Frontend Code (React Native)
Message Handling: The messages state keeps track of the conversation. Each message contains the message content, its sender (user or bot), and a unique ID.

1. Message Handling:

const [messages, setMessages] = useState([]);
const [inputMessage, setInputMessage] = useState('');

2. Sending Queries: When the user types a message and presses the send button, the app sends the user query to the FastAPI backend.

Send Message:

const handleSendMessage = async () => {
    if (inputMessage.trim() === '') return;

    const userQuery = {
        id: Date.now(),
        text: inputMessage.trim(),
        sender: 'user',
    };

    try {
        setMessages((prevMessages) => [...prevMessages, userQuery]);
        setInputMessage("");
        await processUserQuery(inputMessage);
    } catch (error) {
        console.error('Error sending message:', error);
    }
};

3. Rendering Responses: Once the backend sends a response, the frontend displays it as a chat message. The response includes HTML content, which is rendered using the react-native-render-html library.

Rendering Messages:

const renderMessage = (item) => (
    <View
        key={item.id}
        style={[
            styles.messageContainer,
            item.sender === 'user' ? styles.userMessage : styles.botMessage,
        ]}
    >
        {item.sender === 'bot' ? (
            <RenderHTML contentWidth={width} source={{ html: item.text }} />
        ) : (
            <View>
                <RenderHTML contentWidth={width} source={{ html: `<p>${item.text}</p>` }} />
            </View>
        )}
    </View>
);

4. Handling Input: The input field allows users to type messages. The input is cleared once the message is sent.

Input Field:

<TextInput
    ref={inputRef}
    style={styles.input}
    value={inputMessage}
    onChangeText={setInputMessage}
    placeholder="Type your message..."
    multiline={false}
    placeholderTextColor="#aaa"
/>

5. Displaying Send Button: The send button triggers the handleSendMessage function, which sends the user's query to the backend.

Send Button:

<TouchableOpacity onPress={handleSendMessage}>
    <View style={styles.sendButton}>
        <RenderHTML contentWidth={50} source={{ html: "<strong>Send</strong>" }} />
    </View>
</TouchableOpacity>

Deployment Instructions

Make sure the Docker is install and running.
Run the docker compose command from root project

docker compose up --build

Conclusion

In this project, we've built a robust RAG system that uses FastAPI for backend processing and React Native for frontend interaction. By combining Ollama's language models with ChromaDB for vector storage, we've enabled efficient retrieval and query processing. The system allows users to upload PDFs, query them, and receive detailed responses with citations for verification.

Complete source code : https://github.com/vcse59/GenerativeAI/tree/rag-based-application
Youtube Video : https://youtu.be/3OQ0dQ2YqGk?si=IwvBwMhK02sC0exU

Creating an NPM Package with a Sample Library and App Project

Vivek Yadav — Thu, 27 Feb 2025 07:04:11 +0000

1. What is an NPM Package?

An NPM package is a reusable module of code that can be easily shared, installed, and used in different JavaScript projects. It can contain utility functions, classes, components, or any logic that you want to make reusable.

Libraries: Functions or utilities that solve common problems.
Tools: Command-line tools or build utilities.
React Components: Reusable React UI components.

2. Creating the NPM Package

We will create a simple package called simple-math-lib that provides basic math operations like addition and multiplication.

Step 1: Create a Directory for Your Package

Start by creating a new directory for your package:

mkdir simple-math-lib
cd simple-math-lib

Step 2: Initialize the NPM Project

Run npm init to initialize the package:

npm init

You’ll be prompted for details such as:

Package name (e.g., simple-math-lib)
Version (e.g., 1.0.0)
Description (e.g., A simple math library with basic operations)
Entry point (usually index.js)

After answering these questions, package.json will be created.

Step 3: Create the Library Code (index.js)

Inside the package folder, create an index.js file with the following code:

// simple-math-lib/index.js

/**
 * Adds two numbers.
 * @param {number} a - The first number
 * @param {number} b - The second number
 * @returns {number} The sum of a and b
 */
function add(a, b) {
    return a + b;
}

/**
 * Multiplies two numbers.
 * @param {number} a - The first number
 * @param {number} b - The second number
 * @returns {number} The product of a and b
 */
function multiply(a, b) {
    return a * b;
}

// Exporting the functions to be used by others
module.exports = {
    add,
    multiply
};

This file contains two functions: add() and multiply(), and exports them for use by others.

Step 4: Add a README File

Create a README.md to explain the package:

# simple-math-lib

A simple math library with basic operations like addition and multiplication.

## Installation

npm install simple-math-lib

Usage

const math = require('simple-math-lib');

console.log(math.add(2, 3)); // Output: 5
console.log(math.multiply(2, 3)); // Output: 6


---

## **3. Publishing the NPM Package**

### **Log into NPM**

Before publishing your package, you need to log into your **NPM account**. This allows you to publish packages under your account.

Run the following command to log into NPM:

npm login

3. Publishing the NPM Package

Log into NPM

Before publishing your package, you need to log into your NPM account. This allows you to publish packages under your account.

Run the following command to log into NPM:

npm login

You'll be prompted for your NPM username, password, and email address. If you don’t have an NPM account yet, you can create one at https://www.npmjs.com/signup.

Step 1: Publish the Package

To publish your package, run the following command:

npm publish

This will upload the package to the NPM registry, making it available for others to install and use.

You will see a success message like this:

+ simple-math-lib@1.0.0

Step 2: Updating the Package
If you make changes to your package, you will need to bump the version in package.json. For example, to change the version to 1.0.1:

npm version patch  # For minor changes

Then publish again:

npm publish

4. Using the NPM Package in an App

Step 1: Create a New App Project

Now, let's create a simple app that uses the package.

First, create a new directory for your app project:

mkdir my-app
cd my-app

Step 2: Initialize the App Project

Run the following command to initialize the project:

npm init -y

Step 3: Install Your NPM Package

You can install your NPM package by running:

npm install simple-math-lib

This will add simple-math-lib as a dependency in your app.

Step 4: Use the Package in Your App

Create an index.js file in your app project and use the math library:

// my-app/index.js
const math = require('simple-math-lib');

console.log("Addition:", math.add(5, 10));      // 15
console.log("Multiplication:", math.multiply(5, 10)); // 50

Step 5: Run Your App

Run the app:

node index.js

You should see the output:

Addition: 15
Multiplication: 50

5. Conclusion

You’ve now created and published an NPM package, and used it in a separate app. Here's a summary of what we've done:

Created the NPM package: We created a math library called simple-math-lib.
Published it to NPM: We published the package to NPM for others to install and use.
Used it in an app: We created a simple app and used the math functions in it.

Optimizing Chip Placement on Circular Wafers Using Breadth-First Search (BFS)

Vivek Yadav — Tue, 25 Feb 2025 04:22:22 +0000

Introduction

Semiconductor manufacturing involves fabricating microchips on circular silicon wafers. Since microchips (or dies) are rectangular, an efficient arrangement is needed to maximize yield and minimize waste. This article explores how Breadth-First Search (BFS) can be used to optimize chip placement, defect detection, and exposure path planning in wafer lithography.

The Challenge of Chip Placement on Circular Wafers

1. Maximizing Yield

The goal is to fit the maximum number of chips on the wafer while ensuring no overlap and minimum wastage at the wafer edges.

2. Avoiding Defects

Some sections of the wafer may contain defects due to process variations. BFS can help identify and isolate defective regions.

3. Optimizing Exposure Paths

Lithography machines expose patterns onto the wafer in a systematic scanning pattern. BFS can be used to find the most efficient exposure sequence, reducing machine movement and time.

Using BFS for Optimal Chip Placement

Approach:

We model the wafer as a graph, where:

Each node represents a possible chip position.
Edges connect adjacent chip positions.
BFS is used to explore valid placements, starting from the center and expanding outward layer by layer.

Algorithm for BFS Chip Placement:

Start from the center of the wafer (optimal die placement begins at the core).
Expand outward using BFS, checking valid placements.
Ensure no chip extends beyond the circular boundary.
Continue placing until no more valid positions are found.

C++ Implementation:

#include <iostream>
#include <queue>
#include <vector>
#include <cmath>
using namespace std;

struct Point {
    int x, y;
};

void bfs_chip_placement(int wafer_radius, int chip_size) {
    queue<Point> q;
    vector<Point> placements;
    q.push({0, 0});
    vector<vector<bool>> visited(2 * wafer_radius + 1, vector<bool>(2 * wafer_radius + 1, false));

    while (!q.empty()) {
        Point p = q.front();
        q.pop();

        int x = p.x, y = p.y;
        if (x * x + y * y > wafer_radius * wafer_radius || visited[x + wafer_radius][y + wafer_radius])
            continue;

        placements.push_back({x, y});
        visited[x + wafer_radius][y + wafer_radius] = true;

        int directions[4][2] = {{chip_size, 0}, {-chip_size, 0}, {0, chip_size}, {0, -chip_size}};
        for (auto d : directions)
            q.push({x + d[0], y + d[1]});
    }

    cout << "Chip Placements:\n";
    for (auto p : placements)
        cout << "(" << p.x << ", " << p.y << ")\n";
    cout << "Total chip placements on wafer with radius " << wafer_radius << " are " << placements.size() << " with size of " << chip_size << endl; 
}

int main() {
    int wafer_radius = 50;  // Example wafer radius
    int chip_size = 5;      // Example chip size
    bfs_chip_placement(wafer_radius, chip_size);
    return 0;
}

How This Algorithm Works

1. Graph Representation

The wafer is modeled as a grid.
Each valid chip position is treated as a graph node.
BFS expands chip placement outward, ensuring no overlaps.

2. Boundary Check

Ensures that no chip placement exceeds the circular wafer boundary.

3. Efficient Traversal

BFS systematically expands outward, guaranteeing maximum wafer utilization.

BFS for Defect Detection

Defects in wafers can cause clusters of faulty dies. BFS can be used to identify defective regions and mark them for exclusion.

Approach:

Treat the wafer as a graph, where each die is a node.
Use BFS to detect connected defective regions.
Isolate and prevent defective chip placement.

BFS for Lithography Exposure Path Optimization

In photolithography, exposure must follow a systematic pattern to reduce laser movement and increase throughput. BFS can optimize the scanning path.

Approach:

Model the wafer as an exposure grid.
Use BFS to determine the most efficient exposure sequence.
Expand in a spiral or concentric pattern for efficiency.

Conclusion

Breadth-First Search (BFS) is a powerful technique in semiconductor lithography for:

Chip placement optimization (maximizing wafer utilization).
Defect detection and isolation (ensuring quality control).
Efficient lithography exposure paths (minimizing processing time).

By leveraging BFS, manufacturers can reduce waste, improve yield, and optimize wafer production, making it an essential tool in modern chip fabrication.

Cuckoo Hashing: An Efficient Collision Resolution Technique with O(1) Lookup Time

Vivek Yadav — Tue, 25 Feb 2025 03:58:52 +0000

Overview

Cuckoo Hashing is an advanced technique used to resolve hash collisions efficiently. Unlike traditional hashing methods, which may degrade to O(n) in the worst case, Cuckoo Hashing ensures a worst-case time complexity of O(1) for lookups, insertions, and deletions.

Concept of Cuckoo Hashing

Cuckoo Hashing employs a "kick-out" strategy when a collision occurs during insertion. It utilizes two separate hash tables (h1 and h2) and two hash functions (f1(x) and f2(x)). When inserting a data element d1 into h1, if a collision occurs, the existing element in h1 is displaced (kicked out) and moved to h2 based on the second hash function. This process repeats between h1 and h2 until an empty position is found.

To prevent infinite loops during insertion, a predefined constant (max_hash_loop) is used to limit reinsertions. If this limit is reached, the hash tables are resized, and a rehashing operation is performed to accommodate new elements while maintaining O(1) insertion time complexity.

Key Features:

Utilizes two hash tables for efficient collision resolution.
Employs two distinct hash functions to determine data placement.
Supports fast insertion, search, and deletion operations.
Implements a rehashing mechanism to handle infinite insertion loops.

Implementation of Cuckoo Hashing in C++

#include <iostream>
#include <vector>
#include <cstdlib>
#include <ctime>

using namespace std;

#define TABLE_SIZE 11  // Initial hash table size
#define MAX_REHASH 10  // Maximum reinsertion attempts before rehashing

class CuckooHashTable {
private:
    vector<int> table1, table2;
    vector<bool> occupied1, occupied2;

    int hash1(int key) { return key % TABLE_SIZE; }
    int hash2(int key) { return (key / TABLE_SIZE) % TABLE_SIZE; }

    void rehash() {
        cout << "Rehashing required!" << endl;
        vector<int> oldTable1 = table1, oldTable2 = table2;
        vector<bool> oldOccupied1 = occupied1, oldOccupied2 = occupied2;

        int newSize = TABLE_SIZE * 2;
        table1.assign(newSize, -1);
        table2.assign(newSize, -1);
        occupied1.assign(newSize, false);
        occupied2.assign(newSize, false);

        for (int i = 0; i < oldTable1.size(); i++) {
            if (oldOccupied1[i]) insert(oldTable1[i]);
            if (oldOccupied2[i]) insert(oldTable2[i]);
        }
    }

public:
    CuckooHashTable() {
        table1.assign(TABLE_SIZE, -1);
        table2.assign(TABLE_SIZE, -1);
        occupied1.assign(TABLE_SIZE, false);
        occupied2.assign(TABLE_SIZE, false);
    }

    void insert(int key) {
        int pos1 = hash1(key);
        int pos2 = hash2(key);
        int loopCounter = 0;
        int currKey = key;

        while (loopCounter < MAX_REHASH) {
            if (!occupied1[pos1]) {
                table1[pos1] = currKey;
                occupied1[pos1] = true;
                return;
            }
            swap(currKey, table1[pos1]);

            if (!occupied2[pos2]) {
                table2[pos2] = currKey;
                occupied2[pos2] = true;
                return;
            }
            swap(currKey, table2[pos2]);

            pos1 = hash1(currKey);
            pos2 = hash2(currKey);
            loopCounter++;
        }

        rehash();
        insert(currKey);
    }

    bool search(int key) {
        return (occupied1[hash1(key)] && table1[hash1(key)] == key) ||
               (occupied2[hash2(key)] && table2[hash2(key)] == key);
    }

    void remove(int key) {
        if (occupied1[hash1(key)] && table1[hash1(key)] == key) {
            occupied1[hash1(key)] = false;
            table1[hash1(key)] = -1;
            return;
        }
        if (occupied2[hash2(key)] && table2[hash2(key)] == key) {
            occupied2[hash2(key)] = false;
            table2[hash2(key)] = -1;
            return;
        }
        cout << "Key not found!" << endl;
    }

    void display() {
        cout << "Table 1: ";
        for (int i = 0; i < TABLE_SIZE; i++)
            cout << (occupied1[i] ? to_string(table1[i]) : "-") << " ";
        cout << "\nTable 2: ";
        for (int i = 0; i < TABLE_SIZE; i++)
            cout << (occupied2[i] ? to_string(table2[i]) : "-") << " ";
        cout << "\n";
    }
};

int main() {
    CuckooHashTable hashTable;
    hashTable.insert(10);
    hashTable.insert(20);
    hashTable.insert(30);
    hashTable.insert(25);
    hashTable.insert(35);
    hashTable.insert(40);
    hashTable.insert(50);

    hashTable.display();

    cout << "Searching 25: " << (hashTable.search(25) ? "Found" : "Not Found") << endl;
    cout << "Searching 100: " << (hashTable.search(100) ? "Found" : "Not Found") << endl;

    hashTable.remove(30);
    hashTable.display();
    return 0;
}

Explanation:

Data Structure Overview:

Two separate hash tables (table1 and table2).
Boolean vectors (occupied1 and occupied2) track occupied slots.

Hash Functions:

hash1(key) = key % TABLE_SIZE
hash2(key) = (key / TABLE_SIZE) % TABLE_SIZE

Insertion Process:

Try inserting in table1. If occupied, swap with the existing element.
Try inserting the displaced element in table2. If occupied, continue swapping.
If a reinsertion loop is detected (MAX_REHASH reached), rehashing is triggered.

Search and Deletion:

Searching checks both tables.
Deletion marks the slot as unoccupied.

Conclusion

Cuckoo Hashing provides an efficient solution for resolving hash collisions with a guaranteed worst-case O(1) lookup time. By using two hash tables and a kick-out mechanism, it ensures high performance and avoids long probing sequences. The rehashing mechanism allows dynamic resizing when insertion loops occur, maintaining efficiency even in dynamic workloads. This method is particularly useful in applications where quick access time is critical, such as caches, networking applications, and real-time systems.

References:

https://en.wikipedia.org/wiki/Cuckoo_hashing

Using Ollama models (FastAPI + React Native)

Vivek Yadav — Sat, 08 Feb 2025 23:47:32 +0000

What is Ollama

Ollama is a powerful, open-source tool that allows you to run large language models (LLMs) entirely on your local machine, without relying on cloud-based services. It provides an easy way to download, manage, and run AI models with optimized performance, leveraging GPU acceleration when available.

Key Features:

✅ Run LLMs Locally – No internet required after downloading models.
✅ Easy Model Management – Download, switch, and update models effortlessly.
✅ Optimized for Performance – Uses GPU acceleration for faster inference.
✅ Private & Secure – No data leaves your machine.
✅ Custom Model Support – Modify and fine-tune models for specific tasks.
✅ Simple API & CLI – Interact with models programmatically or via command line.

How It Works:

Install Ollama – A simple install command sets it up.
Pull a Model – Example: ollama pull mistral to download Mistral-7B.
Run a Model – Example: ollama run mistral to start interacting.
Integrate with Code – Use the API for automation and app development.

Create a API microservice to interact with Ollama models

We'll use FastAPI to create a microservice that interacts with Ollama models.

FastAPI Code : Ollama.py

Start the API microservice

uvicorn Ollama:app --host 0.0.0.0 --port 8000

Output in Postman:

Create a react native chat bot to call API microservice to process user query

Now, let's build a React Native chatbot that will communicate with the API microservice.

Main Chatbot UI : App.js

Chat Interface : ChatbotUI.js

Start the react native application

# npm install
# npm run web

Output :

Output can be watched at Video

Conclusion

Building a chatbot using Ollama models provides a powerful and private AI experience by running large language models locally. By integrating Ollama with a FastAPI microservice and a React Native frontend, we created a seamless, interactive chatbot that processes user queries efficiently.

This approach offers:
✅ Full control over AI models without cloud dependencies.
✅ Optimized performance using GPU acceleration when available.
✅ Enhanced privacy, as no data is sent to external servers.

Whether you're developing an AI assistant, a customer support bot, or experimenting with LLMs, this setup provides a strong foundation for further improvements and customization. 🚀

Complete code can be found at GitHub

Deep Dive into Server-Sent Events (SSE)

Vivek Yadav — Sat, 08 Feb 2025 06:17:24 +0000

What is SSE ?

Server-Sent Events (SSE) is a mechanism that allows the server to push real-time updates to the client over a single HTTP connection. Unlike WebSockets, which support bidirectional communication, SSE is unidirectional—the server sends data, and the client listens.

1. How SSE Works

Step-by-Step Flow

The client (browser or frontend app) creates an EventSource connection to an SSE endpoint.
The server keeps the connection open and streams data as events.
The client automatically handles reconnections if the connection is lost.
The server sends data in a special format where each event starts with data: and ends with \n\n.

Example SSE Data Format

data: {"message": "Hello World"}

The browser treats this as an event and triggers onmessage

2. SSE vs. WebSockets vs. Long Polling

Feature	SSE	WebSockets	Long Polling
Connection	One-way (server → client)	Two-way (server ↔ client)	Repeated requests
Performance	Efficient (persistent connection)	Best for interactive apps	Inefficient (many requests)
Reconnection	Automatic	Manual handling needed	New request needed
Use Cases	Notifications, updates, logs, AI streaming	Chat, gaming, collaboration apps	Legacy real-time updates

3. When to Use SSE?

SSE is ideal for: ✅ Live notifications (e.g., stock updates, sports scores)
✅ Real-time analytics dashboards
✅ Live AI model responses (e.g., streaming chatbot messages)
✅ IoT device monitoring
✅ Event-driven architectures (e.g., order tracking updates)

Not ideal for:
❌ Bidirectional communication (Use WebSockets)
❌ Very high-frequency updates (e.g., 1000s/sec)

4. Advanced SSE Features

A. Custom Events in SSE

Instead of only using onmessage, SSE supports named events:

Server (FastAPI)

@app.get("/custom_events")
async def custom_events():
    async def event_stream():
        for i in range(5):
            await asyncio.sleep(2)
            yield f"event: update\ndata: {json.dumps({'message': f'Update {i+1}'})}\n\n"
        yield "event: done\ndata: {}\n\n"  # Custom event when completed
    return StreamingResponse(event_stream(), media_type="text/event-stream")

Client (JavaScript)

const eventSource = new EventSource("http://127.0.0.1:8000/custom_events");

eventSource.addEventListener("update", (event) => {
    console.log("Update Event:", JSON.parse(event.data));
});

eventSource.addEventListener("done", () => {
    console.log("Streaming completed.");
    eventSource.close();
});

B. Filtering SSE Data for Specific Users

Server

@app.get("/user_stream")
async def user_stream(user_id: str):
    async def event_stream():
        for i in range(5):
            await asyncio.sleep(2)
            yield f"data: {json.dumps({'user_id': user_id, 'message': f'Update {i+1}'})}\n\n"
    return StreamingResponse(event_stream(), media_type="text/event-stream")

Client

const eventSource = new EventSource("http://127.0.0.1:8000/user_stream?user_id=123");
eventSource.onmessage = (event) => {
    console.log("Received:", JSON.parse(event.data));
};

✅ Now, only events meant for user_id=123 are received.

C. Securing SSE with Authentication

SSE doesn’t support headers for authentication in EventSource. Instead, use:

URL token authentication
Cookie-based authentication
JWT Authentication with FastAPI Dependencies

Example: JWT Authentication

from fastapi import Depends

def verify_token(token: str = Depends(authenticate_user)):  # Your JWT auth function
    return token

@app.get("/secure_events")
async def secure_events(token: str = Depends(verify_token)):
    return StreamingResponse(event_stream(), media_type="text/event-stream")

✅ Now, only authenticated users can access the SSE stream.

D. Optimizing SSE Performance

Use gzip compression: Reduces payload size.
Limit open connections: Too many SSE connections may overload the server.
Use Redis Pub/Sub: For handling multiple clients with event broadcasting.

Using Redis for Broadcasting Events

import aioredis

redis = await aioredis.from_url("redis://localhost")

@app.get("/events")
async def sse_endpoint():
    async def event_stream():
        async with redis.pubsub() as pubsub:
            await pubsub.subscribe("updates")
            while True:
                message = await pubsub.get_message(ignore_subscribe_messages=True)
                if message:
                    yield f"data: {message['data'].decode()}\n\n"

    return StreamingResponse(event_stream(), media_type="text/event-stream")

✅ Now, multiple clients receive the same live updates.

5. Conclusion

🚀 SSE is a lightweight, efficient solution for real-time updates when only server-to-client communication is needed.
💡 Use SSE for live dashboards, notifications, streaming responses, and AI model outputs.
🔧 For bidirectional communication, consider WebSockets.

Understanding DOM Manipulation in JavaScript: A Beginner’s Guide

Vivek Yadav — Fri, 17 Jan 2025 13:15:48 +0000

If you're new to JavaScript, one of the first powerful tools you'll encounter is DOM manipulation. It enables you to interact dynamically with web pages, transforming static content into interactive and dynamic experiences. In this article, we'll break down the fundamentals of DOM manipulation to help you get started on your JavaScript journey.

What is the DOM?

The Document Object Model (DOM) is a programming interface for web documents. It represents the structure of your HTML document as a tree of objects that you can programmatically access and manipulate.

Imagine your HTML document as a family tree:

The root element is the tag.
Inside , you have and as children.
These, in turn, have their own children, like <title>, <div>, or <p> tags.

Using JavaScript, you can:

Access elements in the DOM.
Modify their content, attributes, or styles.
Add, remove, or rearrange elements.

Why Learn DOM Manipulation?

Dynamic Content: Update text, images, and other content based on user interactions.
Interactive Features: Add features like modals, dropdowns, or sliders.
Event Handling: Respond to user actions like clicks, mouse movements, or keyboard input.

How to Access the DOM

The first step in DOM manipulation is selecting elements. JavaScript provides several methods to do this:

1. By ID

Select an element by its id:

const element = document.getElementById('myId');

2. By Class Name

Select elements by their class:

const elements = document.getElementsByClassName('myClass');

3. By Tag Name

Select elements by their tag:

const paragraphs = document.getElementsByTagName('p');

4. Using CSS Selectors

For more flexibility, use querySelector or querySelectorAll:

const firstElement = document.querySelector('.myClass'); // First match
const allElements = document.querySelectorAll('.myClass'); // All matches

Common DOM Manipulations

1. Changing Content

Modify the text or HTML of an element.

- Text Content:

document.getElementById('example').textContent = 'New Text';

- HTML Content:

document.getElementById('example').innerHTML = '<b>Bold Text</b>';

2. Changing Attributes

You can update or add attributes to an element:

const image = document.querySelector('img');
image.setAttribute('src', 'newImage.jpg');
image.setAttribute('alt', 'New Description');

3. Modifying Styles

Use the .style property to change an element’s appearance:

const box = document.getElementById('box');
box.style.backgroundColor = 'blue';
box.style.color = 'white';

Alternatively, manipulate CSS classes:

box.classList.add('active');
box.classList.remove('inactive');
box.classList.toggle('highlight');

4. Adding and Removing Elements

Create new elements dynamically and add them to the DOM:

- Create and Append Elements:

const newDiv = document.createElement('div');
newDiv.textContent = 'I am a new div!';
document.body.appendChild(newDiv);

- Remove Elements:

const element = document.getElementById('removeMe');
element.remove();

Event Handling

DOM manipulation becomes even more powerful when combined with event listeners. An event listener responds to user actions like clicks, mouse movements, or keyboard input.

Example: Button Click

<button id="clickMe">Click Me</button>
<script>
  const button = document.getElementById('clickMe');
  button.addEventListener('click', () => {
    alert('Button was clicked!');
  });
</script>

Common Events:

click: When an element is clicked.
mouseover: When the mouse is over an element.
keydown: When a key is pressed.

Traversing the DOM

Navigating the DOM helps you work with related elements. Some useful properties:

.parentNode or .parentElement: Access the parent of an element.
.children or .childNodes: Access the children of an element.
.nextElementSibling or .previousElementSibling: Access siblings.

Example:

const parent = document.getElementById('parent');
const firstChild = parent.firstElementChild;
const lastChild = parent.lastElementChild;

Performance Tips

1. Minimize DOM Access: Repeatedly accessing the DOM can be slow. Store references to elements in variables when needed multiple times.
2. Batch Updates: Use documentFragment to add multiple elements at once.
3. Avoid Excessive innerHTML: It’s less secure and can lead to performance issues. Use createElement for dynamic content.

Putting It All Together: A Simple Example

Here’s an interactive example that combines several DOM manipulation techniques:

<!DOCTYPE html>
<html>
<head>
  <style>
    .highlight { background-color: yellow; }
  </style>
</head>
<body>
  <h1 id="title">Hello, World!</h1>
  <button id="changeText">Change Text</button>
  <button id="highlight">Highlight</button>

  <script>
    // Change the text of the heading
    document.getElementById('changeText').addEventListener('click', () => {
      document.getElementById('title').textContent = 'You changed the text!';
    });

    // Toggle the highlight class
    document.getElementById('highlight').addEventListener('click', () => {
      document.getElementById('title').classList.toggle('highlight');
    });
  </script>
</body>
</html>

Conclusion

DOM manipulation is a cornerstone of modern web development. By understanding how to access, modify, and interact with elements, you can create dynamic and engaging user experiences. Start with small projects, and as you grow more comfortable, explore frameworks like React or Vue, which build on these fundamentals to streamline DOM interactions.