DEV Community: Somil Gupta

I would love to get some feedback on this.

Somil Gupta — Mon, 23 Dec 2024 23:57:19 +0000

Understanding Amazon Bedrock's New Feature - "Flows"

Somil Gupta for AWS Community Builders ・ Nov 29

Understanding Amazon Bedrock's New Feature - "Flows"

Somil Gupta — Fri, 29 Nov 2024 13:12:24 +0000

Amazon Bedrock continues to evolve its capabilities with the introduction of Flows, a powerful new feature that enables developers to orchestrate complex AI workflows efficiently. This post will explore Flows, how they work, and how you can leverage them in your applications.

What are Amazon Bedrock Flows?

Flows are a new orchestration tool within Amazon Bedrock that allows developers to create sequences of connected events and actions. Think of it as a visual workflow builder for AI operations, where you can chain together different services and define how they interact.

PREREQUISITES

Basic experience navigating the AWS Management Console.
Fundamental understanding of AWS Bedrock service, including AI Agents, Prompts, and Knowledge Bases.

Let's try to create a basic application using Flow Builder - customer_service_flow. In this application, we will take the user input, and based on that, we will decide whether they want to book a new appointment or have a query that we can answer based on our FAQs knowledge base.

After creating a flow, let's examine the types of nodes available and how to use them to create our desired application.

All the nodes in the flow builder are bucketed into six categories:

Logic - Nodes to control the logic of your flow.
Orchestration - Nodes for LLM agents and prompts, helps use a specific prompt or call a particular agent with an input.
Code - A simple node triggers the Lambda function and gets the output.
Data - Everything related to data retrieval, storage, and sending a query to a knowledge base.
AI Services - An Amazon Lex node sends the input to an Amazon Lex bot for interpretation.

Firstly, we will start with a basic prompt node to help us classify whether the user wants to book an appointment or if the query is related to something else. Prompt we will be using:

Take the user {{input}} and analyze whether they want to book an 
appointment or have any other query.

Output "APPOINTMENT" if they want to book a new service appointment 
else, output "OTHER"

Only respond with a category.

Our flow structure will look like this.

Let's enhance our flow by adding some logic to it with the help of the "Condition" node. After classification, this node will point the flow in that respective direction. Think of it as a traffic controller that directs conversations down different paths depending on their content.
For example, after classifying incoming messages, we'll create two distinct paths: one for appointment requests and another for general inquiries. Let's test this with the message "Do you guys do plumbing work?" Since this is a general service inquiry rather than an appointment request, our prompt node classifies it as "OTHER." The Condition node then routes it to "FlowOutputNode_2" where we display the prompt output (for demonstration purposes).

Add our knowledge base and AI agent to the flow to handle the respective route. After doing so, our flow will look something like the below diagram.

And YES, that's it. We have successfully created our desired application using the new flow builder.

Flows for Amazon Bedrock makes it easy to link foundation models (FMs), prompts, and other AWS services to quickly create, test, and run your flows. You can manage flows using the visual builder in the Amazon Bedrock console, which saves a lot of time when creating a generative AI workflow.

Do try it out on your own. Happy Building!!

Thanks for reading my story. If you want to read more stories like this, I invite you to follow me.
Till then, Sayonara! I wish you the best in your learning journey.

How I Used Bedrock Agents to Create a Tool — Medium2Markdown

Somil Gupta — Sun, 15 Sep 2024 00:11:26 +0000

Recently, I faced a problem when I was creating my personal blog which was using Markdown for all my written content. Every blog was on Medium, and it was taking a lot of time to convert those blogs to Markdown files. Hence, I worked on this project.

This application is a simple way to generate markdown files using your blogs on Medium. This tool provides a solution for fetching HTML content from URLs and converting it to Markdown format using AWS Bedrock agents with any FM of your choice. It consists of two main components:

A Flask-based API for fetching HTML content
A Next.js application (using App Router) that handles the Bedrock integration for converting HTML to Markdown

Step 1 — Flask API for HTML Fetching

We are using Selenium to scrape the HTML from the webpage because the Medium ‘GET’ call for a medium-story page returns a partial result that doesn’t contain complete story content. So we render the page in headless Chrome, wait for it to load, and then get the HTML page. This will allow us to get the complete story in the HTML file.

Below is the simple code using Selenium and Flask to create a simple API that takes in the URL and returns the HTML body.



from flask import Flask, jsonify, request
from selenium import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService
from selenium.webdriver.chrome.options import Options
from webdriver_manager.chrome import ChromeDriverManager
from bs4 import BeautifulSoup
import time

app = Flask(__name__)


def get_website_html(url):
    # Set up headless Chrome options
    chrome_options = Options()
    # chrome_options.add_argument("--headless")
    chrome_options.add_argument("--no-sandbox")
    chrome_options.add_argument("--disable-dev-shm-usage")

    # Initialize the WebDriver with ChromeDriverManager
    driver = webdriver.Chrome(service=ChromeService(ChromeDriverManager().install()), options=chrome_options)
    driver.get(url)

    print("Pausing for 5 seconds to allow the page to load...")
    time.sleep(5)  # Pause for 5 seconds

    # Get the HTML content of the page
    html_content = driver.page_source

    # Close the WebDriver
    driver.quit()

    return html_content


@app.route('/get_html', methods=['POST'])
def get_html():
    print('Fetching the HTML content of a website...')

    # Get the URL from the request body
    data = request.get_json()
    if not data or 'url' not in data:
        return jsonify({'error': 'URL is required in the request body'}), 400

    url = data['url']
    html_content = get_website_html(url)

    # Parse the HTML content with BeautifulSoup
    soup = BeautifulSoup(html_content, 'html.parser')

    # Return the HTML content as a JSON response
    return jsonify({'html': str(soup)})


if __name__ == '__main__':
    app.run(debug=True)

You can host this flask server using an AWS EC2 instance or any other cloud provider. Just expose this API with some authorization for security reasons.

After we have the HTML, the next step is to create beautiful markdown content using the HTML body content. We will use Bedrock agent for this, with Claude as the FM.

Step 2 — Setup AWS Bedrock "Agent"

I have created a basic agent using the Bedrock console with the below prompt. You can modify the prompt further for better results.



You are a helpful assistant who takes HTML as input and then parses it 
and returns the blog as a Markdown blog, and the blog should just contain 
the main content body without the title and subtitle. 
Also, remove the first image of the article from the markdown body, 
as we are putting that in the header.

* In the main content markdown, just keep the main body, remove the title 
and subtitle published date, etc.

On top of the markdown, add these things:
* decide on the title and description of the content
* categories can be travel or engineering
* remove the title and description from the main markdown body
* The image will be the first URL of the markdown blog

Sample to put on the top of the markdown
---
title: The Time When I Got Scammed in Georgia
description: A Reminder to Dodge Scams… Or Collect Them Like Souvenirs?
image: /images/blog/blog-post-4.1.png
date: 2024/6/28
authors:
  - nomadic_bug
categories:
  - travel
---

This prompt will help us get the beautiful markdown file in our desired format.

Step 3 — Next.js App with Bedrock Integration

I used Next.js with an app router to create the basic UI for this project. Below is the primary API to run the agent we have created earlier on AWS Bedrock.

The complete code is available here.



// Outline of what we are doing.
// Initialize Bedrock Agent Client with AWS credentials
Initialize BedrockAgentClient with:
    region = "us-east-1"
    access_key = AWS_ACCESS_KEY_ID from environment
    secret_key = AWS_SECRET_ACCESS_KEY from environment

// Set up Agent details
agent_id = "your-agent-id"
agent_alias_id = "your-agent-alias-id"

// Function to invoke Bedrock Agent
Function InvokeBedrockAgent(session_id, input_text):
    Create new InvokeAgentCommand with:
        agent_id = agent_id
        agent_alias_id = agent_alias_id
        session_id = session_id
        input_text = input_text
    Send command to BedrockAgentClient
        Return the completion from the response

// Main API Handler
Function HandlePostRequest(request):
    Extract message and session_id from request body

    If message is missing OR session_id is missing:
        Return error response:
            status = 400
            message = "Please provide both a message and a sessionId."
    response = InvokeBedrockAgent(session_id, message)
        Return success response:
            status = 200
            data = {
                response: response,
                sessionId: session_id
            }

The complete code is available on my Github.

Conclusion

This HTML Fetcher and Markdown Converter is a prototype project that converts web content into easily readable and editable Markdown format. My goal was to make this work, and it does. Some improvements can be made, but this project gave me an idea of how to start.

Thanks for reading my story. If you want to read more stories like this, I invite you to follow me.

Till then, Sayonara! I wish you the best in your learning journey.

Configuring Multiple Ports With Nginx Reverse Proxies on the Same Domain

Somil Gupta — Mon, 12 Aug 2024 18:25:59 +0000

Recently, while working on one of the projects, I was stuck in a situation where I wanted to run two applications on the same server and then consume the application using two different ports.
Something like this:

> Fowarding http://example.com:PORTA -> 127.0.0.1:8080

> Fowarding http://example.com:PORTB -> 127.0.0.1:8081

Usually, when I have one server and one port mapping, I go to Ngrok because of its hassle-free setup. But this time, it was a new challenge.

After some research, I found out that:

Free ngrok.io subdomains: These only work with port 80 (standard HTTP).
Ngrok's TLS "reserved" domains:
- These are better for using non-standard HTTP ports.
- When you set one up, ngrok assigns you a random port.
- This port is different from your local service's port.
Separate domains: The HTTP tunnel (port 80) and the custom port tunnel will have different domain names. You can't use one domain for both.
Free account limitations:
- With a free ngrok account, you can only use the standard HTTP tunnel on port 80.
- You cannot use TLS "reserved" domains or expose services on non-standard ports without upgrading to a paid plan.

This setup means that if you need to expose services on multiple ports or use custom domains, you'll need to upgrade to a paid ngrok plan. With a free account, you're limited to a single HTTP tunnel on the standard web port. In short, it was not possible.

After losing hope with Ngrok, I started looking into Nginx w/ multiple ports. And was finally able to achieve it. Let's dive into the approach.

Understanding Nginx as a reverse proxy:

When NGINX proxies a request, it sends it to a specified proxied server, fetches the response, and sends it back to the client. This makes it an ideal solution for exposing multiple applications running on different ports.

location /some/path/ {
    proxy_pass http://www.example.com/link/;
}

Let's start with the Nginx setup and then move on to the configuration for multiple applications.

Install Nginx:

# For Amazon Linux (AL2)
sudo yum install nginx

# For Ubuntu
sudo apt-get update
sudo apt-get install nginx

Start and enable Nginx:

sudo systemctl start nginx
sudo systemctl enable nginx
sudo systemctl status nginx

For monitoring logs:

sudo tail -f /var/log/nginx/error.log
sudo tail -f /var/log/nginx/access.log

Now, let's configure Nginx to act as a reverse proxy for multiple applications:

http {
    server {
        listen 7233;
        server_name example.com;

        location / {
            proxy_pass http://127.0.0.1:8080;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }
    }

    server {
        listen 8233;
        server_name example.com;

        location / {
            proxy_pass http://127.0.0.1:8081;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }
    }
}

Here's what this configuration does:

It sets up two server blocks, each listening on a different port (7233 and 8233). Each server block forwards all traffic (location /) to a different local port (8080 and 8081 respectively).
The proxy_pass directive specifies where the traffic should be forwarded.

To use this configuration:

Save this configuration to /etc/nginx/nginx.conf or include it in your main Nginx configuration file.
Ensure that your applications are running on 127.0.0.1:8080 and 127.0.0.1:8081.
Test the Nginx configuration: sudo nginx -t If the test passes, reload Nginx:

sudo systemctl reload nginx

Conclusion

We've configured Nginx as a reverse proxy to forward traffic from two external ports to two separate internal applications.
This setup exposes multiple services through a single server, enhancing flexibility and security.
Using Nginx, we've created a scalable solution that can be easily extended to accommodate more applications.

Thanks for reading my story. If you want to read more stories like this, I invite you to follow me.
Till then, Sayonara! I wish you the best in your learning journey.

When I Tackled Rate Limiting Using AWS Step Functions and Lambda

Somil Gupta — Fri, 09 Aug 2024 17:37:27 +0000

In the world of cloud computing, managing API request rates efficiently can make or break your application's performance. Today, I'll share my journey of implementing a robust rate-limiting solution using AWS Step Functions and Lambda.

I have been working with external API calls for a while and have noticed they can sometimes fail for various reasons, such as network issues, server downtime, or rate limits on the server. So, I have built this solution to have a robust system to tackle this problem.

In this solution, we will leverage the AWS Step Function and Lambda Functions to construct a reliable retry mechanism. The State Machine will consist of a collection of Lambda functions invoked and stitched together to produce results. This article will walk you through the step-by-step guide.

The main objective we are trying to solve:

While Step Functions inherently support retries within tasks, our specific challenge involves handling API rate limits from the server we are communicating with. The server imposes a rate limit and responds with a 429 status code if too many requests are made from the same IP address within a short period.

For the simplicity of the architecture, I have shown one retry using two lambda functions; this can be increased easily during implementation.

Architecture Overview

Workflow Explanation

User Invokes Step Function State Machine: The process begins when a user initiates the step function state machine. This could be triggered through an API call, a scheduled event, or another AWS service.
Step Function Invokes Lambda (1st Attempt): The step function invokes the first Lambda function (Lambda 1). This Lambda function is responsible for making the API call.
Response: Status: Lambda 1 Executes the API call and returns a status response. This response indicates whether the API call was successful (e.g., status code 200) or failed (e.g., any status code other than 200).
If Failure Status ≠ 200 (2nd Attempt): If the response from Lambda 1 indicates a failure (status code not equal to 200), the step function will proceed to invoke a retry mechanism. This could involve retrying the same Lambda function or invoking a different Lambda function (Lambda 2) to handle the retry attempt.
Response: Status: Lambda 2 It attempts to execute the API call and returns a status response. Similar to the first Attempt, this response will indicate whether the retry was successful.
If Success Status = 200: If either Lambda 1 or Lambda 2 Successfully executes the API call and returns a status code of 200, the step function completes successfully, and the user is notified of the success.
If Failure Even After Retries: Then we will fail the step function and forward the API error to the user with the appropriate status code.

To explain the architecture easily, I have created the above diagram with one retry only, but we will build the solution with two retries. Below is the state machine diagram.

Step-by-Step Guide

Create a base lambda orchestrator function:
This lambda function will help us in orchestrating the state machine. Executing the state machine and handling logic based on the execution status.
Create a function URL for the lambda function:
Now that the lambda function is ready, we can set up a function URL to trigger/send a request to the lambda function using it. Refer to the article below to turn any lambda function into an API with a function URL.

How to use AWS Lambda to trigger “any” script as an API call | by Somil Gupta | Technology Hits | Medium

Somil Gupta ・ Nov 10, 2023 ・
Medium

Create child lambda functions: These will be simple lambda functions acting as a proxy; they will not handle any logic.

We have to create the same three lambda functions using the step_function_child_lambda code.

Define Step Function State Machine: Next, we'll create a Step Functions state machine with a retry mechanism. Here is an example definition in JSON.

Complete code to implement this is available here:

somilg050 / aws_lambda_functions

Testing the State Machine

Trigger the state machine execution using the first lambda function URL and monitor it through the AWS State Machine Console. You should see the retries and the final result, whether it succeeds or fails.

Conclusion —

Implementing a robust API retry mechanism using AWS Step Functions and Lambda is a powerful way to enhance the reliability of your external unreliable API integrations. I have worked too much with the vendor APIs, and their reliability is something you can not trust. They have rate limits, server IP-based wait times, and so on. This retry using different lambda functions will give us different server URLs, preventing IP-based wait time blocking plus the retry mechanism. I hope my experience inspires you to explore innovative solutions for your own cloud computing challenges.

Robust API Retry Mechanism with AWS Step Functions and Lambda

Somil Gupta — Wed, 29 May 2024 12:31:15 +0000

RAG Application using AWS Bedrock and LangChain

Somil Gupta — Sat, 06 Apr 2024 15:16:55 +0000

Hello, good folks!!
In this part of building the RAG application series, we will leverage Mistral's new model Large using AWS Bedrock and LangChain framework to query over the pdfs.
In the previous article of the series, we learned to build an RAG application using AWS Bedrock and LlamaIndex. To learn more about "what RAG is", please refer to the below article.

Learn to Build a Basic RAG Application | by Somil Gupta | AWS in Plain English

End-to-end Guide Using AWS Bedrock and LlamaIndex to Query Over Your Own PDFs

aws.plainenglish.io

Let's get the learning started.

The implementation of this application involves three components:

1. Create a Vector Store

Load -> Transform -> Embed

We will be using the FAISS vector database, which uses the Facebook AI Similarity Search (FAISS) library. There are many excellent vector store options that you can use, such as ChromaDB or LanceDB.

2. Query Vector Store and 'Retrieve Most Similar.'

The way to handle this is at query time, embed the unstructured query and retrieve the embedding vectors that are 'most similar' to the embedded query. A vector store takes care of storing embedded data and performing vector search for you.

3. Frame the response using LLM and 'Enhanced Context'

Response Generation Using LLM (Large Language Model): Once the relevant documents are retrieved from Vector Store, a large language model uses the information from these documents to generate a coherent and contextually appropriate response.
These three steps clearly explain the application we are going to build now.

First and foremost, we will set up our AWS SDK for Python using Boto3 and AWS CLI. If you have not installed them before -



(base) ➜  ~ pip3 install boto3
(base) ➜  ~ pip3 install awscli

(base) ➜  ~ aws configure

In this example, we'll use the AWS Titan Embeddings model to generate embeddings. You can use any model that generates embeddings.



import boto3

# Load the Bedrock client using Boto3.
bedrock = boto3.client(service_name='bedrock-runtime')

from langchain_community.embeddings.bedrock import BedrockEmbeddings
titan_embeddings = BedrockEmbeddings(model_id="amazon.titan-embed-text-v1",
                                     client=bedrock)

Now, we will set up the Vector Store to store and retrieve embeddings. We have our PDF stored in the "data" folder of the root directory.

In this case we'll split our documents into chunks of 1000 characters with 200 characters of overlap between chunks. The overlap helps mitigate the possibility of separating a statement from an important context related to it.
We will leverage RecursiveCharacterTextSplitter from LangChain, which will recursively split the document using common separators like new lines until each chunk is the appropriate size.
We can embed and store all of our document splits in a single command using the FAISS vector store and titan embedding model.



# Vector Store for Vector Embeddings
from langchain_community.vectorstores.faiss import FAISS

# Imports for Data Ingestion
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.document_loaders.pdf import PyPDFDirectoryLoader

# Load the PDFs from the directory
def data_ingestion():
    loader = PyPDFDirectoryLoader("data")
    documents = loader.load()
    # Split the text into chunks
    text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000,
                                                   chunk_overlap=200)
    docs = text_splitter.split_documents(documents)
    return docs

# Vector Store for Vector Embeddings
def setup_vector_store(documents):
    # Create a vector store using the documents and the embeddings
    vector_store = FAISS.from_documents(
        documents,
        titan_embeddings,
    )
    # Save the vector store locally
    vector_store.save_local("faiss_index")

The next step is to import and load the LLM via Bedrock.



# Import Bedrock for LLM
from langchain_community.llms.bedrock import Bedrock

# Load the LLM from the Bedrock
def load_llm():
    llm = Bedrock(model_id="mistral.mistral-large-2402-v1:0", 
                    client=bedrock, model_kwargs={"max_tokens": 512})
    return llm

We will be using LangChain PromptTemplate to create the prompt template for our LLM. We will produce an answer using a prompt that includes the question and the retrieved data (context).



from langchain.prompts import PromptTemplate

# Create a prompt template
prompt_template = """Use the following pieces of context to answer the 
question at the end. Please follow the following rules:
1. If the answer is not within the context knowledge, kindly state 
that you do not know, rather than attempting to fabricate a response.
2. If you find the answer, please craft a detailed and concise response 
to the question at the end. Aim for a summary of max 250 words, ensuring
 that your explanation is thorough.

{context}

Question: {question}
Helpful Answer:"""

PROMPT = PromptTemplate(template=prompt_template, 
                            input_variables=["context", "question"])

Now, let's write the actual application logic. We want to create a simple application that takes a user question, searches for documents relevant to that question, passes the retrieved documents and initial question to a model, and returns an answer.
We need to define the LangChain Retriever interface. Load RetrievalQA from LangChain as it provides a simple interface for interacting with the LLM.



from langchain.chains.retrieval_qa.base import RetrievalQA

# Create a RetrievalQA chain and invoke the LLM
def get_response(llm, vector_store, query):
    retrieval_qa = RetrievalQA.from_chain_type(
        llm=llm,
        chain_type="stuff",
        retriever=vector_store.as_retriever(
            search_type="similarity", search_kwargs={"k": 3}
        ),
        chain_type_kwargs={"prompt": PROMPT},
        return_source_documents=True,
    )
    return retrieval_qa

Let's put it all together into a chain. This tutorial will use Streamlit to create a UI that interacts with our RAG.

We will provide a simple button in the sidebar to create and update a vector store and store it in the local storage.
Whenever a user enters a query, we will first get the faiss_index from our local storage and then query our LLM using the retrieved context.



def streamlit_ui():
    st.set_page_config("My Gita RAG")
    st.header("RAG implementation using AWS Bedrock and Langchain")

    user_question = st.text_input("Ask me anything from My Gita e.g. 
                                          What is the meaning of life?")

    with st.sidebar:
        st.title("Update Or Create Vector Embeddings")

        if st.button("Update Vector Store"):
            with st.spinner("Processing..."):
                docs = data_ingestion()
                setup_vector_store(docs)
                st.success("Done")

    if st.button("Generate Response") or user_question:
        # first check if the vector store exists
        if not os.path.exists("faiss_index"):
            st.error("Please create the vector store 
                                first from the sidebar.")
            return
        if not user_question:
            st.error("Please enter a question.")
            return
        with st.spinner("Processing..."):
            faiss_index = FAISS.load_local("faiss_index", 
                                          embeddings=titan_embeddings,
                                      allow_dangerous_deserialization=True)
            llm = load_llm()
            st.write(get_response(llm, faiss_index, user_question))
            st.success("Done")

This is how our Streamlit application will look.

The complete code for the application is available here on my github: somilg050.

You can play around with the code by customizing the prompt and changing the parameters to the LLM.

In conclusion, we have created an application that takes a question, retrieves relevant documents, constructs a prompt, passes that to a model, and parses the output.

Thanks for reading the tutorial. I hope you learn something new today. If you want to read more stories like this, I invite you to follow me.

Till then, Sayonara! I wish you the best in your learning journey.

DEV Community: Somil Gupta

I would love to get some feedback on this.

Understanding Amazon Bedrock's New Feature - "Flows"

Somil Gupta for AWS Community Builders ・ Nov 29

Understanding Amazon Bedrock's New Feature - "Flows"

What are Amazon Bedrock Flows?

How I Used Bedrock Agents to Create a Tool — Medium2Markdown

Step 1 — Flask API for HTML Fetching

Step 2 — Setup AWS Bedrock "Agent"

Step 3 — Next.js App with Bedrock Integration

Conclusion

Configuring Multiple Ports With Nginx Reverse Proxies on the Same Domain

Understanding Nginx as a reverse proxy:

Conclusion

When I Tackled Rate Limiting Using AWS Step Functions and Lambda

The main objective we are trying to solve:

Step-by-Step Guide

How to use AWS Lambda to trigger “any” script as an API call | by Somil Gupta | Technology Hits | Medium

Somil Gupta ・ Nov 10, 2023 ・ Medium

somilg050 / aws_lambda_functions

Testing the State Machine

Conclusion —

Robust API Retry Mechanism with AWS Step Functions and Lambda

RAG Application using AWS Bedrock and LangChain

Learn to Build a Basic RAG Application | by Somil Gupta | AWS in Plain English

Somil Gupta ・ Nov 10, 2023 ・
Medium