DEV Community

Cover image for Project GenAI Url Digest
Koti eswar mani gudi
Koti eswar mani gudi

Posted on

Project GenAI Url Digest

Project Description:-

GenAI-Url-Digest is a simple GenAI-RAG web app built with the LangChain framework and deployed in GKE as a microservice, including both frontend and backend. We improved observability with OpenTelemetry distributed tracing and set up continuous integration with Jenkins pipelines and continuous deployment with Argo CD. Ansible configured the Jenkins master node, and Terraform deployed the infrastructure. A pgvector database stores vector embeddings, allowing LangChain to perform vector searches. The project supports both local and cloud deployment.

Tools Used

  • Langchain Framework with RAG
  • Google Kubernetes Engine
  • Opentelemetry Tracing
  • Ansible
  • Terraform
  • Jenkins
  • Docker Compose

Website Images

                        Website Architecture
Enter fullscreen mode Exit fullscreen mode

Image description
Url Post Route
Image description

                     Question & Answer page
Enter fullscreen mode Exit fullscreen mode

Image description

                Cloud trace dashboard
Enter fullscreen mode Exit fullscreen mode

Image description

Here is the small intro to project.For full project setup visit below github repo once.

PROJECT_GITHUB_REPO
**
Sample Docker Compose File for Local Deployment **

name: "urldigest"
version: '3.8'

services:
  frontend:
    build: ./src/frontend
    environment:
      - BACKEND_SERVER_URL=http://backend:5001
      - REDIS_SERVER_URL=redis
      - REDIS_AUTH_STRING=gkem1234
      - OTLP_ENDPOINT=otelcol

    depends_on:
      - backend
    ports:
      - "5000:5000"
    networks:
    - frontend 
    - backend 
    - redis
    - otelcol

  backend:
    build: ./src/backend
    ports:
      - "5001:5001"
    environment:
      - GOOGLE_APPLICATION_CREDENTIALS=/run/secrets/key.json
      - PGVECTOR_HOST_URL=pgvector
      - PGVECTOR_USER=gkem
      - PGVECTOR_PASSWORD=gkem1234
      - PGVECTOR_DATABASE=urldigest
      - OTLP_ENDPOINT=otelcol
    secrets:
      - key.json
    depends_on:
      - redis
      - pgvector
      - otelcol
    networks:
    - backend 
    - otelcol
    - pgvector

  redis:
    image: redis:latest
    command: redis-server /usr/local/etc/redis/redis.conf --requirepass gkem1234
    volumes:
      - ./config/redis.conf:/usr/local/etc/redis/redis.conf
    ports:
      - 6379:6379
    networks:
    - redis

  otelcol:
    image: otel/opentelemetry-collector:latest
    command: --config /etc/otelcol/config.yaml
    volumes:
      - ./collector.yaml:/etc/otelcol/config.yaml
    ports:
      - 4317:4317
      - 4318:4318
      - 55681:55681
      - 55680:55680
    depends_on:
      - zipkin
    networks:
    - otelcol
    - zipkin

  pgvector:
    image: pgvector/pgvector:pg16
    environment:
      - POSTGRES_USER=gkem
      - POSTGRES_PASSWORD=gkem1234
      - POSTGRES_DB=urldigest
    ports:
      - 5432:5432
    networks:
    - pgvector

  zipkin:
    image: openzipkin/zipkin
    ports:
      - 9411:9411
    networks:
      - zipkin

secrets:
  key.json:
    file: ./key.json

networks:
  zipkin:
  redis:
  frontend:
  backend:
  otelcol:
  pgvector:

Enter fullscreen mode Exit fullscreen mode

Image of Timescale

🚀 pgai Vectorizer: SQLAlchemy and LiteLLM Make Vector Search Simple

We built pgai Vectorizer to simplify embedding management for AI applications—without needing a separate database or complex infrastructure. Since launch, developers have created over 3,000 vectorizers on Timescale Cloud, with many more self-hosted.

Read more →

Top comments (0)

Postmark Image

Speedy emails, satisfied customers

Are delayed transactional emails costing you user satisfaction? Postmark delivers your emails almost instantly, keeping your customers happy and connected.

Sign up