AI-Driven Content Retrieval with PostgreSQL, Ollama and Open-Source models

Hasnain — Fri, 08 Nov 2024 06:34:50 +0000

This is a submission for the Open Source AI Challenge with pgai and Ollama

What I Built

This platform leverages advanced semantic search algorithms, including Ollama LLM, Langchain, and LangGraph, to analyze and retrieve relevant data from a diverse content repository. After processing the input prompt, the system performs a semantic search using pgvector to fetch pertinent content, which serves as references for the language model to enhance output accuracy and relevance.

The platform efficiently handles various input formats such as images, YouTube links, and PDF files. For video content, metadata is fetched via the YouTube API and then summarized using Ollama LLM to provide concise and relevant overviews. Similarly, the system is equipped to summarize content from images, ensuring a comprehensive and versatile user experience across media types.

In addition to semantic processing, the platform includes image generation through the Stable Diffusion algorithm, enriching the interaction by providing visual content based on textual prompts. Generated images are stored securely in Pinata (IPFS), and Postgres with pgAI integration supports a robust chat history feature, enabling seamless continuity and easy referencing of prior conversations.

Ollama LLM:

import { ChatOllama } from "@langchain/ollama";
const llm = new ChatOllama({
    model: "llama3",
    baseUrl: process.env.OLLAMA_BASE_URL,
  });

Vector Embedding model:

async function getOllamaEmbedding(text: string): Promise<number[]> {
 const embeddings = new OllamaEmbeddings({
      model: "mxbai-embed-large", 
      baseUrl: "http://localhost:11434",
    });
    const vectors = await embeddings.embedQuery(text);

    console.log("Embedding from Ollama:", vectors);
    return vectors;
}

Vector Search:

import pg from "pg";
const { Pool } = pg;
export const pool = new Pool({
  user: process.env.PG_USER,
  host: process.env.PG_HOST,
  database: process.env.PG_DATABASE,
  password: process.env.PG_PASSWORD,
  port: process.env.PG_PORT,
});
export async function vectorSearch(message: string) {
  const embedding = await getOllamaEmbedding(message);
  const embeddingString = `[${embedding.join(",")}]`;

  const result = await pool.query(
    `
    SELECT id, email,content,
    (embedding <-> $1::vector) AS distance
    FROM fileTable
    ORDER BY distance
    LIMIT 5;
    `,
    [embeddingString]
  );
  return result.rows;
}

Store File content into vectorstore:

export async function setEmbedding(
  combinedContent: Document[],
  email: string
) {
  try {
    const embeddings = new OllamaEmbeddings({
      model: "mxbai-embed-large", 
      baseUrl: "http://localhost:11434", 
    });
    const vectors = await embeddings.embedDocuments(
      combinedContent.map((chunk) => chunk.pageContent.replace(/\n/g, " "))
    );

    const query = `
    INSERT INTO fileTable (email, embedding,content)
    VALUES ($1, $2::vector, $3)
  `;
    for (let i = 0; i < combinedContent.length; i++) {
      await pool.query(query, [
        email,
        vectors[i], // embedding for the current chunk
        combinedContent[i].pageContent, // original content
      ]);
    }
  } catch (err) {
    console.error("Failed to store question:", err);
  }
}

Demo

Hasnain01-hub / mediafusion

AI Chat Assistant

Description

This platform utilizes advanced semantic search algorithms to analyze and retrieve relevant data from this diverse content repository. After processing the input prompt, the system performs a semantic search to fetch pertinent content, which is then used as references for the language model to enhance the accuracy and relevance of its outputs.

The platform is designed to handle inputs across a diverse formats including images, YouTube links, and pdf files. For videos, it fetches metadata through the YouTube API, which is then effectively summarized using a LLM to provide concise and relevant content overviews. Similarly, the system is equipped to summarize content from images, ensuring a comprehensive and versatile user experience that accommodates various types of media.

Additionally, the platform incorporates the capability to generate images using the Stable Diffusion algorithm, further enriching the user interaction experience by providing visual content generation based on textual prompts…

View on GitHub


Youtube Video Summarizer


Rag Search

Tools Used

pgvector: pgvector stores the embeddings for content across various formats, including text, pdf. It supports similarity-based searches directly within PostgreSQL, enabling efficient and accurate content retrieval. Feature: Used in the vector search function to retrieve the closest matching content based on embeddings.
pgAI: pgAI enhances PostgreSQL with AI-driven capabilities, such as processing similarity scores and providing feedback. It integrates directly within SQL queries to score or rank content relevance, improving the quality of search results. Feature: Applied in querying and sorting results based on similarity distances to optimize semantic search accuracy.
Ollama: Ollama is used for generating text embeddings that represent content meaning and for summarizing media files. It supports various tasks, including embedding generation for content similarity and text summarization for complex media formats. Feature: Used in both the embedding model and language model functions to process and summarize text and video content effectively.

Final Thoughts

In summary, this platform seamlessly integrates advanced semantic search and multimedia processing capabilities, powered by tools like Ollama LLM, Langchain, and LangGraph. With robust support from pgvector and pgAI for vector-based search and retrieval, it handles diverse media inputs, enhances content relevancy, and offers visually rich interactions through Stable Diffusion. The platform’s comprehensive architecture ensures continuity, user engagement, and a smooth, versatile experience across formats.

Prize Categories:
Open-source Models from Ollama, Vectorizer Vibe, All the Extensions

Thank you for taking the time to explore my project!

Real-Time Chat Application with React and Pinata for Effortless Messaging and File Sharing

Hasnain — Wed, 09 Oct 2024 08:36:26 +0000

This is a submission for the The Pinata Challenge

What I Built

I developed a real-time chat application with key features, and it is fully responsive, making it accessible from mobile UI as well.
features:

Real-Time Messaging: The application supports real-time messaging, ensuring that messages sent by users instantly appear in the chat room without delays.
File Sharing: The application allows users to upload and share various file types, such as images, PDFs, and documents, using Pinata for secure and decentralized file storage.
Chat Rooms: Users have the ability to create, join, and leave chat rooms, making the platform dynamic and user-controlled.
Message History: Each chat room maintains a message history, which is stored and made visible to users when they join a room, allowing for continuity in conversations.
Google User Authentication: Users can log in and log out seamlessly using their Google accounts, enabled via Firebase Authentication.
Private Messaging: Users can also send private messages to other users for one-on-one conversations.

Demo

Working: app link

My Code

Hasnain01-hub / realtime-chat-app

Project Title - 2Connect

Features:

Google User Authentication: Users can be able to log in and log out using their Google accounts via Firebase Authentication.
Chat Rooms: Users should be able to create, join, and leave chat rooms.
Real-Time Messaging: Messages sent by users should appear in the chat room in real time.
Share File: It allows users to upload and share various types of files, including images and PDFs using Pinata IPFS.
Message History: The chat application should store the history of messages for each chat room, which should be visible to users when they join the room.
Private Messaging: Users should be able to send private messages to other users.

Tech Stack

React JS
Pinata
Firebase Firestore
Firebase Realtime database
Firebase Authentication

Screenshots


Chat Screen


Create Group


Group Chat


Join Group


Login Page

View on GitHub

More Details

Users can share images directly by selecting them from their local directory. The application utilizes Pinata IPFS to manage and store these assets in a secure and decentralized manner.


Chat window

Users can also share PDFs, which are embedded in an iframe, allowing them to view and download the file directly from the chat interface.


Group chat

Users can join multiple group forums, allowing them to participate in various discussions and collaborate with different communities. This feature enhances user engagement by providing a platform for diverse interactions, enabling users to share ideas, ask questions, and connect with others folks.


Join Group

TechStack Used:

React JS
Pinata
Firebase Realtime Database
Firebase Firestore
Firebase Authentication

DEV Community: Hasnain

AI-Driven Content Retrieval with PostgreSQL, Ollama and Open-Source models

What I Built

Ollama LLM:

Vector Embedding model:

Vector Search:

Store File content into vectorstore:

Demo

Hasnain01-hub / mediafusion

AI Chat Assistant

Description

Tools Used

Final Thoughts

Real-Time Chat Application with React and Pinata for Effortless Messaging and File Sharing

What I Built

Demo

My Code

Hasnain01-hub / realtime-chat-app

Project Title - 2Connect

Features:

Tech Stack

Screenshots

More Details