DEV Community: Rajesh Kumar

Using LLM, Postgres VectorDB, and OpenAI to Perform Semantic Search on PDF Documents

Rajesh Kumar — Thu, 15 Feb 2024 05:30:37 +0000

Objective

The goal of this project is to build a prototype app that can perform similarity search on PDF documents using VectorSearch (Postgres). The app will use the Langchain llm framework and OpenAI to extract and compare semantic vectors from the PDFs.

TechStack

Postgres with VectorSearch (Run in docker)
OpenAI (Create/Must have OpenAI key) : https://openai.com/
Langchain LLM
Python

Source: https://github.com/rajeshkumarbehura/pdf-reader-search

Keywords to learn

langchain framework, embeedding, vectorsearch or vector database, PVector, Document Loader

Explanation

PDF is a common format for documents in organizations, and it is fascinating to test llm semantic search on PDFs.

We used the book "Teach yourself Java in 21 days" for our testing.

This app is a test case for extracting and embedding PDF content and storing it in a database. We use Postgres as the vector database and DBeaver as the database viewer. The PVector framework handles the data design and embedding process.

Understand Steps -

Extracted the PDF content and split it into a list of documents.
Created a database connection string and used it for PGVector (framework class) to handle the creation of embeddings and push them into the database. If the table did not exist, it created it automatically.
Loaded the documents and their embeddings into the database.
Used Dbeaver tool to view the data and tables.(https://dbeaver.io/)

Wrote question for testing as ask_question(query="What is Incrementing and Decrementing ?")

Note : For more search options, check out the langchain documentation on different methods such as
similarity_search
search
similarity_search_with_score
Do more experiments on this functions and get better understanding.

Execution




Run postgess.yml using docker-compose command.
Update openai_key in Reader.py file
Run main function either to extract pdf file or search query.

Reference

https://www.youtube.com/watch?v=zxo3T4aQj6Q&t=1224s
https://github.com/pgvector/pgvector
https://python.langchain.com/docs/integrations/vectorstores/pgvector
https://python.langchain.com/docs/modules/data_connection/document_loaders/pdf
https://python.langchain.com/docs/integrations/vectorstores/pgembedding

Aws Cognito email Passwordless

Rajesh Kumar — Sat, 10 Apr 2021 15:33:59 +0000

For my project, I was looking for passwordless implementation and I found few ways but most of the ways look a bit confusing for me and took time to resolve the problem. I went through few reference documents, videos and finally summarize the steps, how to set up quickly using serverless & terraform scripts.

Find the code's here :
https://github.com/rajeshkumarbehura/aws-cognito-passwordless

Description

Implementation for user signup and login by email-id using the passwordless concept in AWS Cognito.

Deployment steps

1. deploy the lambda using a serverless framework.
2. deploy AWS Cognito & ses email account using terraform.
3. verify your email id for ses account.
4. set up lambda for Cognito triggers.
5. test the passwordless using AWS Cognito CLI commands.

Note: By default, this project was setup in "ap-southeast-1".
Set SES Email Id with your email id = <<SES_VERIFIED_EMAIL_ID>>
Make sure terraform's tf & serverless.yml files will be updated with the appropriate region and email id.

Check out the code from my github project "aws-cognito-passwordless". The link is provided above.

Installation Requirement :

Serveless must be installed. https://www.serverless.com/framework/docs/getting-started/
Terraform must be installed. https://www.terraform.io/downloads.html
aws cli commands must be installed. https://aws.amazon.com/cli/
Setup aws default credential(aws_access_key_id,aws_secret_access_key) for local pc
All the deployment will happen in AWS account default setting in pc.

1. Deploy lambda

Make sure my github project is cloned.

1. Go to email-passwordless-lambda project.
2. Go to the serverless.yml file and replace <<SES_VERIFIED_EMAIL_ID>> with your email id.
3. Run the command "sls deploy".
4. It will set up 5 lambdas for your aws account.

2. Deploy Aws Cognito & ses email account

1. Go to cognito-setup project folder
2. Go to variables.tf update your aws region and replace <<SES_VERIFIED_EMAIL_ID>> with your email id same as for lambda.
3. Run command "terraform init"
4. Run command "terraform plan"
5. Run command "terraform apply"

3. Ses email account verification

1. After deployment, an email will be sent to your account <<SES_VERIFIED_EMAIL_ID>>. Go to your email account and verify the account. Note- without verification of email, email-passwordless will not work.

4. Setup lambdas for Cognito triggers.

Now you login to AWS console and go to user pools in Cognito, select your own created user-pool and setup Triggers for user-pool as
image link.
https://github.com/rajeshkumarbehura/aws-cognito-passwordless/blob/main/images/cognito-lambda-triggers.png

5. Test passwordless email using aws cli command.

Go to your user pool's App Clients and get your App client Id as in the below image link
https://github.com/rajeshkumarbehura/aws-cognito-passwordless/blob/main/images/app-clients-detail.png

1. User sign-up

<<app-client-id>>
<<SES_VERIFIED_EMAIL_ID>>

Signup the user using cli command in your local system-   
aws cognito-idp sign-up --client-id <<app-client-id>>  --username <<SES_VERIFIED_EMAIL_ID>> -password 12345678

for exmaple - (it's only sample)
aws cognito-idp sign-up --client-id 4cgrq69gatdp03sa7k6  --username rajesh.xxx@gmail.com -password 12345678

2. User sign-in & send token to your email

Sign-in the user using cli command in your local system- 

aws cognito-idp initiate-auth --client-id <<app-client-id>> --auth-flow CUSTOM_AUTH --auth-parameters USERNAME="<<SES_VERIFIED_EMAIL_ID>>",PASSWORD=""

For exmaple - (it's only sample)
aws cognito-idp initiate-auth --client-id 568fqvgq612k636hho        --auth-flow CUSTOM_AUTH --auth-parameters USERNAME="rajesh.xxxxx@gmail.com",PASSWORD=""

Now here, your receive session value in your command line, and you will receive a token in your email account.

3. User token validation

<<SESSION_VALUE_FROM_SINGINE_COMMAND>> = get the value above command
<<TOKEN_FROM_EMAIL>> = get the token from email

Token validation using commandline in your system -

aws cognito-idp respond-to-auth-challenge --client-id <<app-client-id>> --challenge-name CUSTOM_CHALLENGE --challenge-responses ANSWER=<<TOKEN_FROM_EMAIL>>,USERNAME="<<SES_VERIFIED_EMAIL_ID>>"
--session "<<SESSION_VALUE_FROM_SINGINE_COMMAND>>"

for example - (it's only sample)
aws cognito-idp respond-to-auth-challenge --client-id 568fqvgq612k63 --challenge-name CUSTOM_CHALLENGE --challenge-responses ANSWER=780322,USERNAME="rajesh.XXXX@gmail.com"
--session "XXXXXXXXXXXXXXXXXXX....XXX"

After the above command, you will recive jwt token and refresh token on your command line.

References

https://www.youtube.com/watch?v=grTaNCwPj58
https://github.com/aws-samples/amazon-cognito-passwordless-email-auth

Deploy dockerized app in aws ecs with fresh new vpc.

Rajesh Kumar — Mon, 25 May 2020 15:20:30 +0000

After struggling for few days to deploy a dockerrized applicatin in ECS, found the solution & simplified into script, so that anyone can deploy easily simply replacing app folder.

This example is for deployment of App with fresh new VPC. Vpc will be created during ecs/fargate deployment.

git location

https://github.com/rajeshkumarbehura/vpc_ecs_app_deploy

Requirement an application to deploy in Aws ECS

Project must dockerized. In my repo, its demo-app folder which is nestjs based simple application.You can have java/nodejs/python any kind of project as long as it is dockerized.
aws cli & terraform cli must be installed in your command line.

aws --version terraform --version
aws details must be avaialble
export AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXX
export AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXX
export AWS_REGION=ap-southeast-1
export AWS_PROFILE=dev // dev profile must match in terraform/providers.ts file
It will create fresh new VPC for ecs/fargate deployment.
After successfully deployment, terraform will print the alb path as

demoapp_alb_hostname = dev-demoapp-XXXXXXX.ap-southeast-1.elb.amazonaws.com

Steps to deploy

Clone the project into your local.

a. Validate aws profile

export AWS_PROFILE=dev

this dev profile name must match with deploy/terraform/provider.tf

# Specify the provider and access details
provider "aws" {
  shared_credentials_file = "$HOME/.aws/credentials"
  profile                 = "dev"
  region                  = var.aws_region
}

b. Update shell script file

Two shell script files exist inside deploy folder.

create_repo_in_ecr.sh : Update aws profile details in this file. This file will create aws ecr repository for project. And print a link after successulry created. Eg - XXXXXX.dkr.ecr.ap-southeast-1.amazonaws.com/demo_app_repo

This is ECR_REPO_PATH for your project.

push_image_to_ecr_&_deploy_ecs.sh : Update aws profile details & ECR_REPO_PATH. This file will follows below step.
- create docker container,
- container will tagged for ecr repo,
- push into ecr repository
- application deploy into aws ecs from ecr.

c. Execution

Now in final steps, after updating above details, go to deploy folder in command line.

Run create_repo_in_ecr.sh file for creating repo.
Run push_image_to_ecr_&_deploy_ecs.sh to deploy application

d. Destroy in Aws

Cleaning or deleteing all resources from aws, use below steps-

Go to deploy/terraform folder location in command line & run below command

terraform destroy

How to deploy your own application

Replace your application inside demo-app content & make sure your application working dockerized. Please do not change demo-app
Create health check api - api/index. This is configured inside terraform/variable.tf file in terraform folder

variable "demoapp_health_check_path" { default = "/api/index" }

Optional step- to create your all configuration with a new name, do a find all for "demoapp" text inside terraform folder and replace with ur own text. For example- replace "demoapp" text with "customerapp". Then do terraform plan, and check all things are working or not. if any place, please rename according to your text.

Play with terraform, ecr & ecs to explore more.

Deploy dockerized app in Aws ECS in existing vpc.

Rajesh Kumar — Sun, 24 May 2020 12:53:54 +0000

After struggling for few days to deploy a dockerrized applicatin in ECS, found the solution & simplified into script, so that anyone can deploy easily simply replacing app folder.

To deploy your application in ECS,

Your docker container image needs to be push to ECR.
From ECR repo, application will deploy in ECS

git location

https://github.com/rajeshkumarbehura/ecs-app-deploy

Requirement an application to deploy in Aws ECS

Project must dockerized. In my repo, its demo-app folder which is nestjs based simple application.You can have java/nodejs/python any kind of project as long as it is dockerized.
aws cli & terraform cli must be available in your command line. aws --version terraform --version
aws details must be avaialble
AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXX
AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXX
AWS_REGION=ap-southeast-1
AWS_PROFILE=dev
You should know, whats your existing vpc & public subnet name in aws account.

Steps to deploy

Clone the project in your local.

Update VPC details in terraform

Go to /deploy/terraform/network.tf and replace dev-data-stream-vpc name with your existing vpc name.

Replace dev-data-stream-public-subnet name, with your existing public subnet tied with your existing vpc name.

variable "existing_vpc_name" {
  description = "Existing VPC in your aws cloud"
  default     = "dev-data-stream-vpc"
}

variable "existing_vpc_public_subnet_name" {
  description = "public subnet name which tie up with existing vpc"
  default     = "dev-data-stream-public-subnet"
}

Application

go to deploy folder
update AWS parameters in create_repo_in_ecr.sh and run this file.
Above script will create a repo in your aws account and print that ecr repo link in your console.
Copy ecr link & update aws parameter in push_image_to_ecr_&_deploy_ecs.sh file.
finally run push_image_to_ecr_&_deploy_ecs.sh script on your console to deploy this application

Destroy in AWS

Cleaning or deleteing all resources in aws, use below command.

terraform destroy

Once you can deploy and destroy successfully, then you can change more parameter and play around terraform files to understand more.

How to deploy your own application

Replace your application inside demo-app content & make sure your application working dockerized.
Create health check api - api/index This is configured inside variable.tf file in terraform folder
Optional step- to create your all configuration with a new name, do a find all for "demoapp" text inside terraform folder and replace with ur own text. For example- replace "demoapp" text with "customerapp". Then do terraform plan, and check all things are working or not. if any place, please rename according to your text.

Play with terraform, ecr & ecs to explore more.

Kafkajs & NestJS with Typescript Simplified Example

Rajesh Kumar — Sat, 23 May 2020 17:18:17 +0000

This code example tried to simplify kafka integration with nestjs.

Github location

https://github.com/rajeshkumarbehura/ts-nestjs-kafka

How to integrate-

Kafka module is created as global module in this project. To integrate your project, copy app/common/kakfa module and inject KafkaModule as app module.

    KafkaModule.register({
      clientId: 'test-app-client',
      brokers: ['localhost:9092'],
      groupId: 'test-app-group',
    }

In my project, it is injected to top level in app.module.ts

Create A Kafka Payload

Before sending message, create a custom payload to send message

    const payload: KafkaPayload = {
      messageId: '' + new Date().valueOf(), // uuid
      body: message,
      messageType: 'Say.Hello',  
      topicName: 'hello.topic', 
    };

Send message to KafkaTopic

Inject KafkaService to your service or controller and call sendMessage.

const value = await this.kafkaService.sendMessage('hello.topic', payload);

Consumer Implementation

Extend AbstractKafkaConsumer class and implement registerTopic method.
Inside registerTopic, you only need to add topic names.
No need to inject any service, as its implemented globally.

@Injectable()
export class ConsumerService extends AbstractKafkaConsumer {
    protected registerTopic() {
        this.addTopic('hello.topic');  
        this.addTopic('hello.fixed.topic');
    }
}

Subscribe to Topic when GroupId is not fixed

Add annotation to the method and define topic name as its parameter

 @SubscribeTo('hello.topic')
 helloSubscriber(payload: KafkaPayload) {
        console.log('Print message after receiving', payload);
 }

Subscribe to Topic when GroupId is fixed

When multiple containers or apps are running during horizontal scaling and your only one container/application required to listen to topic.

 @SubscribeToFixedGroup('hello.fixed.topic')
 helloSubscriber(payload: KafkaPayload) {
        console.log('Print message after receiving', payload);
 }