DEV Community: Jaydeep Biswas

Seeking advice on optimizing response time and handling multiple requests on AWS instance with NVIDIA A10G GPU

Jaydeep Biswas — Thu, 11 Apr 2024 06:27:42 +0000

Hey everyone,

I'm currently facing some challenges with optimizing the response time of my AWS instance. Here's the setup: I'm using a g5.xlarge instance which houses a single NVIDIA A10G GPU with 24GB of VRAM. Recently, I fine-tuned a mistralai/Mistral-7B-Instruct-v0.2 model on my custom data and then merged it with the base model. Additionally, I applied quantization methods to optimize further.

However, when I send a request to my fine-tuned model, it's taking approximately 3 minutes to respond, even for requests with a max token of 1024. I'm looking for suggestions on how to reduce this response time.

Furthermore, I've encountered errors when attempting to handle multiple requests simultaneously. Specifically, I've received errors like:

"Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument mat2 in method wrapper_CUDA_mm)"
"The SW shall provide an estimated value for the torque CUDA error: device-side assert triggered CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with TORCH_USE_CUDA_DSA to enable device-side assertions."

Could someone please guide me on how to address these errors and efficiently handle multiple requests simultaneously on my AWS instance?

Any help or advice would be greatly appreciated. Thanks in advance!

LangChain's 3rd Module: Agents🦜🕴️

Jaydeep Biswas — Wed, 10 Jan 2024 18:29:33 +0000

Hey there! Throughout our latest blog series, we've delved into a wide array of subjects. Here's an overview of the topics we've explored thus far:

Exploring LangChain's Agents 🔍🤖

Today, I want to dive into this exciting concept called "Agents" ** in LangChain. It's pretty mind-blowing!
**LangChain introduces an innovative idea called "Agents" that takes the concept of chains to a whole new level. Agents use language models to dynamically figure out sequences of actions to perform, making them highly versatile and adaptable. Unlike regular chains, where actions are hardcoded in code, agents utilize language models as reasoning engines to decide which actions to take and in what order.

The Agent is the main part responsible for decision-making. It harnesses the power of a language model and a prompt to figure out the next steps to achieve a specific objective. The inputs to an agent usually include:

Tools: Descriptions of available tools (more on this later).
User Input: The high-level objective or query from the user.
Intermediate Steps: A history of (action, tool output) pairs executed to reach the current user input.

The result of an agent can either be the next thing to do (AgentActions) or the ultimate reply to give to the user (AgentFinish). An action includes details about a tool and the input needed for that tool.

Tools 🛠️

Tools are interfaces that an agent can use to interact with the world. They allow agents to perform various tasks like searching the web, running shell commands, or accessing external APIs. In LangChain, tools are crucial for expanding the capabilities of agents and helping them achieve diverse tasks.

To use tools in LangChain, you can load them using the following code:

from langchain.agents import load_tools

tool_names = [...]
tools = load_tools(tool_names)

Some tools may need a base Language Model (LLM) for initialization. In such cases, you can pass an LLM like this:

from langchain.agents import load_tools

tool_names = [...]
llm = ...
tools = load_tools(tool_names, llm=llm)

This setup allows you to access a variety of tools and integrate them into your agent's workflows. The complete list of tools with usage documentation is available here.

Examples of Tools 📚🔧

DuckDuckGo

The DuckDuckGo tool lets you perform web searches using its search engine. Here's an example:

from langchain.tools import DuckDuckGoSearchRun

search = DuckDuckGoSearchRun()
search.run("Manchester United vs Luton Town match summary")

DataForSeo

The DataForSeo toolkit allows you to get search engine results using the DataForSeo API. To use it, you need to set up your API credentials:

import os

os.environ["DATAFORSEO_LOGIN"] = "<your_api_access_username>"
os.environ["DATAFORSEO_PASSWORD"] = "<your_api_access_password>"

Once credentials are set, you can create a DataForSeoAPIWrapper tool to access the API:

from langchain.utilities.dataforseo_api_search import DataForSeoAPIWrapper

wrapper = DataForSeoAPIWrapper()

result = wrapper.run("Weather in Los Angeles")

The DataForSeoAPIWrapper tool fetches search engine results from various sources.

You can customize the type of results and fields returned in the JSON response:

json_wrapper = DataForSeoAPIWrapper(
    json_result_types=["organic", "knowledge_graph", "answer_box"],
    json_result_fields=["type", "title", "description", "text"],
    top_count=3,
)

json_result = json_wrapper.results("Bill Gates")

Specify the location and language for your search results:

customized_wrapper = DataForSeoAPIWrapper(
    top_count=10,
    json_result_types=["organic", "local_pack"],
    json_result_fields=["title", "description", "type"],
    params={"location_name": "Germany", "language_code": "en"},
)

customized_result = customized_wrapper.results("coffee near me")

Choose the search engine:

customized_wrapper = DataForSeoAPIWrapper(
    top_count=10,
    json_result_types=["organic", "local_pack"],
    json_result_fields=["title", "description", "type"],
    params={"location_name": "Germany", "language_code": "en", "se_name": "bing"},
)

customized_result = customized_wrapper.results("coffee near me")

The search is customized to use Bing as the search engine.

Specify the type of search:

maps_search = DataForSeoAPIWrapper(
    top_count=10,
    json_result_fields=["title", "value", "address", "rating", "type"],
    params={
        "location_coordinate": "52.512,13.36,12z",
        "language_code": "en",
        "se_type": "maps",
    },
)

maps_search_result = maps_search.results("coffee near me")

These examples showcase how you can customize searches based on result types, fields, location, language, search engine, and search type.

Shell (bash)

The Shell toolkit gives agents the ability to interact with the shell environment, allowing them to run shell commands. This feature is powerful but should be used carefully, especially in sandboxed environments. Here's how to use the Shell tool:

from langchain.tools import ShellTool

shell_tool = ShellTool()

result = shell_tool.run({"commands": ["echo 'Hello World!'", "time"]})

In this example, the Shell tool runs two shell commands: echoing "Hello World!" and displaying the current time.

You can provide the Shell tool to an agent for more complex tasks. Here's an example of an agent using the Shell tool to fetch links from a web page:

from langchain.agents import AgentType, initialize_agent
from langchain.chat_models import ChatOpenAI

llm = ChatOpenAI(temperature=0.1)

shell_tool.description = shell_tool.description + f"args {shell_tool.args}".replace(
    "{", "{{"
).replace("}", "}}")
self_ask_with_search = initialize_agent(
    [shell_tool], llm, agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION, verbose=True
)
self_ask_with_search.run(
    "Download the langchain.com webpage and grep for all urls. Return only a sorted list of them. Be sure to use double quotes."
)

In this scenario, the agent uses the Shell tool to execute a series of commands to fetch, filter, and sort URLs from a web page.

The examples provided showcase some of the tools available in LangChain. These tools ultimately expand the capabilities of agents (explored in the next subsection) and empower them to efficiently perform various tasks. Depending on your project's needs, you can choose the tools and toolkits that best suit your requirements and integrate them into your agent's workflows.

Return to Agents ↩️🤖

Let's talk about agents now.

The AgentExecutor is like the engine that runs an agent. It's responsible for calling the agent, making it do actions, giving the agent the results, and doing this in a loop until the agent finishes its task. In simpler terms, it might look something like this:

next_action = agent.get_action(...)
while next_action != AgentFinish:
    observation = run(next_action)
    next_action = agent.get_action(..., next_action, observation)
return next_action

The AgentExecutor deals with various complexities, like what happens when the agent picks a tool that doesn't exist, handling tool errors, managing what the agent produces, and providing logs at different levels.

Although the AgentExecutor class is the main runtime for agents in LangChain, there are other experimental runtimes like:

Plan-and-execute Agent
Baby AGI
Auto GPT

To understand the agent framework better, let's build a basic agent from scratch and then explore pre-built agents.

Before we dive into building the agent, let's review some key terms and schema:

AgentAction: This is like a set of instructions for the agent. It includes the tool to use and tool_input the input for that tool. - AgentFinish: This indicates the agent has finished its task and is ready to give a response to the user. - Intermediate Steps: These are like records of what the agent did before. They help the agent remember context for future actions.

Now, let's create a simple agent using OpenAI Function Calling. We'll start by making a tool that calculates word length. This is useful because language models sometimes make mistakes when counting word lengths due to tokenization.

First, load the language model:

from langchain.chat_models import ChatOpenAI

llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)

Test the model with a word length calculation::

llm.invoke("how many letters in the word educa?")

Define a simple function to calculate word length:

from langchain.agents import tool

@tool
def get_word_length(word: str) -> int:
    """Returns the length of a word."""
    return len(word)

We've created a tool named get_word_length that takes a word as input and returns its length.
Now, create a prompt for the agent. The prompt guides the agent on how to reason and format the output:

from langchain.prompts import ChatPromptTemplate, MessagesPlaceholder

prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You are a very powerful assistant but not great at calculating word lengths.",
        ),
        ("user", "{input}"),
        MessagesPlaceholder(variable_name="agent_scratchpad"),
    ]
)

To provide tools to the agent, format them as OpenAI function calls:

from langchain.tools.render import format_tool_to_openai_function

llm_with_tools = llm.bind(functions=[format_tool_to_openai_function(t) for t in tools])

Create the agent by defining input mappings and connecting components:

from langchain.agents.format_scratchpad import format_to_openai_function_messages
from langchain.agents.output_parsers import OpenAIFunctionsAgentOutputParser

agent = (
    {
        "input": lambda x: x["input"],
        "agent_scratchpad": lambda x: format_to_openai_function_messages(
            x["intermediate_steps"]
        ),
    }
    | prompt
    | llm_with_tools
    | OpenAIFunctionsAgentOutputParser()
)

We've created our agent, which understands user input, uses available tools, and formats output.
Interact with the agent:

agent.invoke({"input": "how many letters in the word educa?", "intermediate_steps": []})

Now, let's write a runtime for the agent. The simplest runtime calls the agent, executes actions, and repeats until the agent finishes:

from langchain.schema.agent import AgentFinish

user_input = "how many letters in the word educa?"
intermediate_steps = []

while True:
    output = agent.invoke(
        {
            "input": user_input,
            "intermediate_steps": intermediate_steps,
        }
    )
    if isinstance(output, AgentFinish):
        final_result = output.return_values["output"]
        break
    else:
        print(f"TOOL NAME: {output.tool}")
        print(f"TOOL INPUT: {output.tool_input}")
        tool = {"get_word_length": get_word_length}[output.tool]
        observation = tool.run(output.tool_input)
        intermediate_steps.append((output, observation))

print(final_result)

To simplify this, use the AgentExecutor class. It encapsulates agent execution and offers error handling, early stopping, tracing, and other improvements:

from langchain.agents import AgentExecutor

agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

agent_executor.invoke({"input": "how many letters in the word educa?"})

The AgentExecutor makes it easier to interact with the agent and simplifies the execution process.

Memory in Agents 🧠🤖

The agent we've made so far doesn't remember past conversations, making it stateless. To enable follow-up questions and continuous conversations, we need to add memory to the agent. Here are the two steps involved:

Add a memory variable in the prompt to store chat history.
Keep track of the chat history during interactions.

Let's start by adding a memory placeholder in the prompt:

from langchain.prompts import MessagesPlaceholder

MEMORY_KEY = "chat_history"
prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You are a very powerful assistant but not great at calculating word lengths.",
        ),
        MessagesPlaceholder(variable_name=MEMORY_KEY),
        ("user", "{input}"),
        MessagesPlaceholder(variable_name="agent_scratchpad"),
    ]
)

Now, create a list to track the chat history:

from langchain.schema.messages import HumanMessage, AIMessage

chat_history = []

In the agent creation step, include the memory as well:

agent = (
    {
        "input": lambda x: x["input"],
        "agent_scratchpad": lambda x: format_to_openai_function_messages(
            x["intermediate_steps"]
        ),
        "chat_history": lambda x: x["chat_history"],
    }
    | prompt
    | llm_with_tools
    | OpenAIFunctionsAgentOutputParser()
)

When running the agent, make sure to update the chat history:

input1 = "how many letters in the word educa?"
result = agent_executor.invoke({"input": input1, "chat_history": chat_history})
chat_history.extend([
    HumanMessage(content=input1),
    AIMessage(content=result["output"]),
])
agent_executor.invoke({"input": "is that a real word?", "chat_history": chat_history})

This lets the agent maintain a conversation history and answer follow-up questions based on past interactions.

Congratulations! You've successfully created and executed your first end-to-end agent in LangChain. To explore LangChain's capabilities further, you can delve into:

Different agent types supported.
Pre-built Agents
How to work with tools and tool integrations.

Agent Types 🤖📝

LangChain offers various agent types, each suited for specific use cases. Here are some available agents:

Zero-shot ReAct: Chooses tools based on their descriptions using the ReAct framework. Versatile and requires tool descriptions.
Structured input ReAct: Handles multi-input tools, suitable for tasks like web browsing. Uses a tools' argument schema for structured input.
OpenAI Functions: Designed for models fine-tuned for function calling, compatible with models like gpt-3.5-turbo-0613 and gpt-4-0613.
Conversational: Tailored for conversational settings, uses ReAct for tool selection, and employs memory to remember previous interactions.
Self-ask with search: Relying on a single tool, "Intermediate Answer," it looks up factual answers to questions.
ReAct document store: Interacts with a document store using the ReAct framework, requiring "Search" and "Lookup" tools.

Explore these agent types to find the one that best suits your needs in LangChain. These agents allow you to bind a set of tools within them to handle actions and generate responses.

Prebuilt Agents 🤖🛠️

Let's continue our exploration of agents, focusing on prebuilt agents available in LangChain.

LangChain Gmail Toolkit 📧🔧

LangChain provides a convenient toolkit for Gmail, allowing you to connect your LangChain email to the Gmail API. To get started, follow these steps:

Set Up Credentials:

Download the credentials.json file as explained in the Gmail API documentation.
Install required libraries using the following commands:

 pip install --upgrade google-api-python-client
 pip install --upgrade google-auth-oauthlib
 pip install --upgrade google-auth-httplib2
 pip install beautifulsoup4  # Optional for parsing HTML messages

Create Gmail Toolkit:

Initialize the toolkit with default settings:

 from langchain.agents.agent_toolkits import GmailToolkit

 toolkit = GmailToolkit()

Customize authentication as needed. Behind the scenes, a googleapi resource is created using the following methods:

 from langchain.tools.gmail.utils import build_resource_service, get_gmail_credentials

 credentials = get_gmail_credentials(
     token_file="token.json",
     scopes=["https://mail.google.com/"],
     client_secrets_file="credentials.json",
 )
 api_resource = build_resource_service(credentials=credentials)
 toolkit = GmailToolkit(api_resource=api_resource)

Use Toolkit Tools:
- The toolkit offers various tools such as GmailCreateDraft, GmailSendMessage, GmailSearch, GmailGetMessage, and GmailGetThread.

_GmailCreateDraft_: Create a draft email with specified message fields.
_GmailSendMessage_: Send email messages.
_GmailSearch_: Search for email messages or threads.
_GmailGetMessage_: Fetch an email by message ID.
_GmailGetThread_: Search for email messages.

Initialize Agent:

Initialize the agent with the toolkit and other settings:

 from langchain.llms import OpenAI
 from langchain.agents import initialize_agent, AgentType

 llm = OpenAI(temperature=0)
 agent = initialize_agent(
     tools=toolkit.get_tools(),
     llm=llm,
     agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,
 )

Examples:

Create a Gmail draft for editing:

 agent.run("Create a Gmail draft for me to edit...")

Search for the latest email in your drafts:

 agent.run("Could you search in my drafts for the latest email?")

These examples demonstrate LangChain's Gmail toolkit capabilities, enabling programmatic interactions with Gmail.

SQL Database Agent 📊🤖

This agent interacts with SQL databases, particularly the Chinook database. Be cautious as it is still in development. To use:

Initialize Agent:

   from langchain.agents import create_sql_agent
   from langchain.agents.agent_toolkits import SQLDatabaseToolkit
   from langchain.sql_database import SQLDatabase
   from langchain.llms.openai import OpenAI
   from langchain.agents import AgentExecutor
   from langchain.agents.agent_types import AgentType
   from langchain.chat_models import ChatOpenAI

   db = SQLDatabase.from_uri("sqlite:///../../../../../notebooks/Chinook.db")
   toolkit = SQLDatabaseToolkit(db=db, llm=OpenAI(temperature=0))

   agent_executor = create_sql_agent(
       llm=OpenAI(temperature=0),
       toolkit=toolkit,
       verbose=True,
       agent_type=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
   )

_Disclaimer_

- The query chain may generate insert/update/delete queries. Be cautious, and use a custom prompt or create a SQL user without write permissions if needed.
- Be aware that running certain queries, such as "run the biggest query possible," could overload your SQL database, especially if it contains millions of rows.
- Data warehouse-oriented databases often support user-level quotas to limit resource usage.

Examples:

Describe a table:

 agent_executor.run("Describe the playlisttrack table")

Run a query:

 agent_executor.run("List the total sales per country. Which country's customers spent the most?")

The agent will execute the query and provide the result, such as the country with the highest total sales.

To get the total number of tracks in each playlist, you can use the following query:

```
agent_executor.run("Show the total number of tracks in each playlist. The Playlist name should be included in the result.")
```

The agent will return the playlist names along with the corresponding total track counts.

Caution:
- Be cautious about running certain queries that could overload your database.

Pandas DataFrame Agent 🐼📊🤖

This agent interacts with Pandas DataFrames for question-answering purposes. Use with caution to prevent potential harm from generated Python code:

Initialize Agent:

   from langchain_experimental.agents.agent_toolkits import create_pandas_dataframe_agent
   from langchain.chat_models import ChatOpenAI
   from langchain.agents.agent_types import AgentType

   from langchain.llms import OpenAI
   import pandas as pd

   df = pd.read_csv("titanic.csv")

   agent = create_pandas_dataframe_agent(OpenAI(temperature=0), df, verbose=True)

Examples:
- Count rows in the DataFrame:
```
 agent.run("how many rows are there?")
```

Filter rows based on criteria:

 agent.run("how many people have more than 3 siblings")

Jira Toolkit 📅🔧

The Jira toolkit allows agents to interact with a Jira instance. Follow these steps:

Install Libraries and Set Environment Variables:

   %pip install atlassian-python-api

   import os
   from langchain.agents import AgentType
   from langchain.agents import initialize_agent
   from langchain.agents.agent_toolkits.jira.toolkit import JiraToolkit
   from langchain.llms import OpenAI
   from langchain.utilities.jira import JiraAPIWrapper

   os.environ["JIRA_API_TOKEN"] = "abc"
   os.environ["JIRA_USERNAME"] = "123"
   os.environ["JIRA_INSTANCE_URL"] = "https://jira.atlassian.com"
   os.environ["OPENAI_API_KEY"] = "xyz"

   llm = OpenAI(temperature=0)
   jira = JiraAPIWrapper()
   toolkit = JiraToolkit.from_jira_api_wrapper(jira)
   agent = initialize_agent(
       toolkit.get_tools(), llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True
   )

Examples:

Create a new issue in a project:

 agent.run("make a new issue in project PW to remind me to make more fried rice")

Now, you can interact with your Jira instance using natural language instructions and the Jira toolkit.

LangChain's 2nd Module: Retrieval🦜🐕

Jaydeep Biswas — Sun, 07 Jan 2024 18:29:28 +0000

In our recent blog series, we've traversed through a diverse range of topics. Here's the topics we've covered so far:

Retrieval Augmented Generation (RAG) is a crucial process in LangChain, especially for applications that need specific user data not present in the model's training set. In simpler terms, it involves fetching external data and blending it seamlessly into the language model's generation process. LangChain offers a robust set of tools and features to make this process easy, accommodating both simple and complex applications.

Let's break down the components involved in the retrieval process:

Document Loaders 📄

Document loaders in LangChain enable the extraction of data from various sources. They enable the extraction of data from various sources, boasting over 100 loaders that support a diverse range of document types and sources such as private S3 buckets, public websites and databases. All these loaders ingest data into Document classes.

You have the flexibility to choose a document loader based on your specific needs from here. Here are examples of different loaders:

Text File Loader

To load a simple .txt file into a document, you can use the TextLoader:

from langchain.document_loaders import TextLoader

loader = TextLoader("./sample.txt")
document = loader.load()

CSV Loader

For loading data from a CSV file, LangChain provides the CSVLoader. You can even customize parsing by specifying field names:

from langchain.document_loaders.csv_loader import CSVLoader

loader = CSVLoader(file_path='./example_data/sample.csv')
documents = loader.load()

loader = CSVLoader(file_path='./example_data/mlb_teams_2012.csv', csv_args={
    'delimiter': ',',
    'quotechar': '"',
    'fieldnames': ['MLB Team', 'Payroll in millions', 'Wins']
})
documents = loader.load()

PDF Loaders

LangChain's PDF Loaders offer various methods for parsing and extracting content from PDF files. Different loaders cater to different needs:

PyPDFLoader (Basic PDF Parsing)

from langchain.document_loaders import PyPDFLoader

loader = PyPDFLoader("example_data/layout-parser-paper.pdf")
pages = loader.load_and_split()

MathPixLoader (Mathematical Content and Diagrams)

from langchain.document_loaders import MathpixPDFLoader

loader = MathpixPDFLoader("example_data/math-content.pdf")
data = loader.load()

PyMuPDFLoader (Fast PDF Parsing with Detailed Metadata)

from langchain.document_loaders import PyMuPDFLoader

loader = PyMuPDFLoader("example_data/layout-parser-paper.pdf")
data = loader.load()

# Optionally pass additional arguments for PyMuPDF's get_text() call
data = loader.load(option="text")

PDFMiner Loader (Granular Control over Text Extraction)

from langchain.document_loaders import PDFMinerLoader

loader = PDFMinerLoader("example_data/layout-parser-paper.pdf")
data = loader.load()

AmazonTextractPDFParser utilizes AWS Textract for OCR and other advanced PDF parsing features.

from langchain.document_loaders import AmazonTextractPDFLoader

# Requires AWS account and configuration
loader = AmazonTextractPDFLoader("example_data/complex-layout.pdf")
documents = loader.load()

PDFMinerPDFasHTMLLoader generates HTML from PDF for semantic parsing.

from langchain.document_loaders import PDFMinerPDFasHTMLLoader

loader = PDFMinerPDFasHTMLLoader("example_data/layout-parser-paper.pdf")
data = loader.load()

PDFPlumberLoader provides detailed metadata and supports one document per page.

from langchain.document_loaders import PDFPlumberLoader

loader = PDFPlumberLoader("example_data/layout-parser-paper.pdf")
data = loader.load()

Integrated Loaders play a vital role in LangChain by allowing direct data loading from various applications such as Slack, Figma, Google Drive, databases, and more. These loaders empower LLMs to seamlessly incorporate information from diverse sources, expanding the capabilities of language generation applications.

Let's explore a couple of examples to illustrate how Integrated Loaders can be employed:

Example I - Slack 💬

Slack, a popular instant messaging platform, can be integrated into LLM workflows with ease. Here's a simplified step-by-step guide:

Go to your Slack Workspace Management page.
Navigate to {your_slack_domain}.slack.com/services/export.
Select the desired date range and initiate the export.
Slack notifies you via email and DM once the export is ready.
The exported data is in a .zip file located in your Downloads folder or the designated download path.

Now, you can use the SlackDirectoryLoader from the langchain.document_loaders package to load this data into LangChain:

from langchain.document_loaders import SlackDirectoryLoader

SLACK_WORKSPACE_URL = "https://xxx.slack.com"  # Replace with your Slack URL
LOCAL_ZIPFILE = ""  # Path to the Slack zip file

loader = SlackDirectoryLoader(LOCAL_ZIPFILE, SLACK_WORKSPACE_URL)
docs = loader.load()
print(docs)

Example II - Figma 🎨

Figma, a widely-used tool for interface design, offers a REST API for data integration. Here's a simplified guide:

Obtain the Figma file key from the URL format: https://www.figma.com/file/{filekey}/sampleFilename.
Node IDs are found in the URL parameter ?node-id={node_id}.
Generate an access token following instructions at the Figma Help Center.

Now, you can use the FigmaFileLoader class from langchain.document_loaders.figma to load Figma data into LangChain. This example demonstrates how to generate HTML/CSS code based on Figma design input:

import os
from langchain.document_loaders.figma import FigmaFileLoader
from langchain.text_splitter import CharacterTextSplitter
from langchain.chat_models import ChatOpenAI
from langchain.indexes import VectorstoreIndexCreator
from langchain.chains import ConversationChain, LLMChain
from langchain.memory import ConversationBufferWindowMemory
from langchain.prompts.chat import ChatPromptTemplate, SystemMessagePromptTemplate, AIMessagePromptTemplate, HumanMessagePromptTemplate

figma_loader = FigmaFileLoader(
    os.environ.get("ACCESS_TOKEN"),
    os.environ.get("NODE_IDS"),
    os.environ.get("FILE_KEY"),
)

index = VectorstoreIndexCreator().from_loaders([figma_loader])
figma_doc_retriever = index.vectorstore.as_retriever()

The generate_code function uses the Figma data to create HTML/CSS code.
It employs a templated conversation with a GPT-based model.

def generate_code(human_input):
    # Template for system and human prompts
    system_prompt_template = "Your coding instructions..."
    human_prompt_template = "Code the {text}. Ensure it's mobile responsive"

    # Creating prompt templates
    system_message_prompt = SystemMessagePromptTemplate.from_template(system_prompt_template)
    human_message_prompt = HumanMessagePromptTemplate.from_template(human_prompt_template)

    # Setting up the AI model
    gpt_4 = ChatOpenAI(temperature=0.02, model_name="gpt-4")

    # Retrieving relevant documents
    relevant_nodes = figma_doc_retriever.get_relevant_documents(human_input)

    # Generating and formatting the prompt
    conversation = [system_message_prompt, human_message_prompt]
    chat_prompt = ChatPromptTemplate.from_messages(conversation)
    response = gpt_4(chat_prompt.format_prompt(context=relevant_nodes, text=human_input).to_messages())

    return response

# Example usage
response = generate_code("page top header")
print(response.content)

In this example, the generate_code function utilizes Figma data to create HTML/CSS code through LangChain's capabilities. These Integrated Loaders showcase how LangChain simplifies the integration of external data, enabling powerful applications in various domains.

Document Transformers 🔄

Document Transformers in LangChain help shape and modify documents, the building blocks we've created earlier. These tools are crucial for tasks like breaking down long texts, combining information, and filtering content, making them fit neatly into a model's understanding or specific application requirements.

One handy tool is the RecursiveCharacterTextSplitter, a versatile text splitter using a character list. It allows you to tweak parameters like chunk size, overlap, and starting index. Here's a simple example in Python:

from langchain.text_splitter import RecursiveCharacterTextSplitter

state_of_the_union = "Your long text here..."

text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=100,
    chunk_overlap=20,
    length_function=len,
    add_start_index=True,
)

texts = text_splitter.create_documents([state_of_the_union])
print(texts[0])
print(texts[1])

Another tool is the CharacterTextSplitter, which divides text based on a chosen character, with controls for chunk size and overlap:

from langchain.text_splitter import CharacterTextSplitter

text_splitter = CharacterTextSplitter(
    separator="\n\n",
    chunk_size=1000,
    chunk_overlap=200,
    length_function=len,
    is_separator_regex=False,
)

texts = text_splitter.create_documents([state_of_the_union])
print(texts[0])

Now, if you're dealing with HTML content, use the HTMLHeaderTextSplitter. It cleverly splits HTML content based on header tags while keeping the semantic structure intact:

from langchain.text_splitter import HTMLHeaderTextSplitter

html_string = "Your HTML content here..."
headers_to_split_on = [("h1", "Header 1"), ("h2", "Header 2")]

html_splitter = HTMLHeaderTextSplitter(headers_to_split_on=headers_to_split_on)
html_header_splits = html_splitter.split_text(html_string)
print(html_header_splits[0])

Things get even more interesting when you combine different tools. For example, combining HTMLHeaderTextSplitter with the Pipelined Splitter:

from langchain.text_splitter import HTMLHeaderTextSplitter, RecursiveCharacterTextSplitter

url = "https://example.com"
headers_to_split_on = [("h1", "Header 1"), ("h2", "Header 2")]
html_splitter = HTMLHeaderTextSplitter(headers_to_split_on=headers_to_split_on)
html_header_splits = html_splitter.split_text_from_url(url)

chunk_size = 500
text_splitter = RecursiveCharacterTextSplitter(chunk_size=chunk_size)
splits = text_splitter.split_documents(html_header_splits)
print(splits[0])

LangChain also offers specialized splitters for different programming languages, such as the Python Code Splitter and the JavaScript Code Splitter:

from langchain.text_splitter import RecursiveCharacterTextSplitter, Language

python_code = """
def hello_world():
    print("Hello, World!")
hello_world()
"""

python_splitter = RecursiveCharacterTextSplitter.from_language(
    language=Language.PYTHON, chunk_size=50
)
python_docs = python_splitter.create_documents([python_code])
print(python_docs[0])

js_code = """
function helloWorld() {
  console.log("Hello, World!");
}
helloWorld();
"""

js_splitter = RecursiveCharacterTextSplitter.from_language(
    language=Language.JS, chunk_size=60
)
js_docs = js_splitter.create_documents([js_code])
print(js_docs[0])

For handling text based on token count (useful for models with token limits), there's the TokenTextSplitter:

from langchain.text_splitter import TokenTextSplitter

text_splitter = TokenTextSplitter(chunk_size=10)
texts = text_splitter.split_text(state_of_the_union)
print(texts[0])

Lastly, there's the LongContextReorder that shuffles documents to prevent performance slowdowns in models due to lengthy contexts:

from langchain.document_transformers import LongContextReorder

reordering = LongContextReorder()
reordered_docs = reordering.transform_documents(docs)
print(reordered_docs[0])

These tools showcase the incredible ways you can transform documents in LangChain, from simple text splitting to complex reordering and language-specific separation. For more detailed insights and specific use cases, diving into the LangChain documentation and Integrations section is highly recommended. And don't worry, in our examples, the loaders have already done the heavy lifting of creating chunked documents for us!

Text Embedding Model 📝➡️🔠

Text Embedding Models in LangChain bring a standardized way of handling various embedding model providers like OpenAI, Cohere, and Hugging Face. These models work by transforming text into vector representations, allowing for powerful operations like semantic search through text similarity in vector space.

Getting started is usually a breeze, involving the installation of specific packages and setting up API keys. In our case, we've already taken care of this for OpenAI.

In LangChain, the go-to method for embedding multiple texts is embed_documents. Take a look at this example using OpenAI:

from langchain.embeddings import OpenAIEmbeddings

# Initialize the model
embeddings_model = OpenAIEmbeddings()

# Embed a list of texts
embeddings = embeddings_model.embed_documents(
    ["Hi there!", "Oh, hello!", "What's your name?", "My friends call me World", "Hello World!"]
)
print("Number of documents embedded:", len(embeddings))
print("Dimension of each embedding:", len(embeddings[0]))

For a single text, like a search query, you can use embed_query. This is handy for comparing a query to a set of document embeddings:

from langchain.embeddings import OpenAIEmbeddings

# Initialize the model
embeddings_model = OpenAIEmbeddings()

# Embed a single query
embedded_query = embeddings_model.embed_query("What was the name mentioned in the conversation?")
print("First five dimensions of the embedded query:", embedded_query[:5])

Understanding these embeddings is key. Each piece of text becomes a vector, and the dimension depends on the model used – for OpenAI, it's typically 1536-dimensions vectors. These embeddings then used for retrieving relevant information.

And here's the cool part: LangChain isn't limited to just OpenAI. It's designed to seamlessly work with various providers. While the setup and usage might differ slightly based on the provider, the core concept of embedding texts into vector space stays the same. For all the nitty-gritty details, including advanced configurations and integrations with different embedding model providers, the LangChain documentation in the Integrations section is a goldmine.

Vector Stores 🗄️

Vector Stores in LangChain are the vaults that efficiently store and search those text embeddings. LangChain integrates with over 50 vector stores, offering a standardized interface for a smooth user experience.

Let's dive into an example where we've embedded texts and now want to store and search them using Chroma:

from langchain.vectorstores import Chroma

db = Chroma.from_texts(embedded_texts)
similar_texts = db.similarity_search("search query")

Alternatively, if we want to use FAISS for creating indexes, here's an example:

from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import FAISS

pdfstore = FAISS.from_documents(pdfpages, 
            embedding=OpenAIEmbeddings())

airtablestore = FAISS.from_documents(airtabledocs, 
            embedding=OpenAIEmbeddings())

Retrievers 🔍

Retrievers in LangChain are like smart search engines, but way more flexible. They don't just find documents, they understand what you're looking for. Unlike vector stores that focus on storing, retrievers are all about finding information.

Let's start with the Chroma retriever. Setting it up involves a few steps, like installing Chroma with pip install chromadb. Then, you load, split, embed, and retrieve documents. Here's a simple example:

from langchain.embeddings import OpenAIEmbeddings
from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores import Chroma

full_text = open("state_of_the_union.txt", "r").read()
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=100)
texts = text_splitter.split_text(full_text)

embeddings = OpenAIEmbeddings()
db = Chroma.from_texts(texts, embeddings)
retriever = db.as_retriever()

retrieved_docs = retriever.invoke("What did the president say about Ketanji Brown Jackson?")
print(retrieved_docs[0].page_content)

Now, there's the MultiQueryRetriever, automates prompt tuning by generating multiple queries for a user input query and combines the results. It generates multiple queries based on your input and combines the results. Check it out:

from langchain.chat_models import ChatOpenAI
from langchain.retrievers.multi_query import MultiQueryRetriever

question = "What are the approaches to Task Decomposition?"
llm = ChatOpenAI(temperature=0)
retriever_from_llm = MultiQueryRetriever.from_llm(
    retriever=db.as_retriever(), llm=llm
)

unique_docs = retriever_from_llm.get_relevant_documents(query=question)
print("Number of unique documents:", len(unique_docs))

Now, imagine you want just the relevant parts from a long document. That's where Contextual Compression Retriever steps in. It compresses retrieved documents, keeping only the relevant info. Take a look:

from langchain.llms import OpenAI
from langchain.retrievers import ContextualCompressionRetriever
from langchain.retrievers.document_compressors import LLMChainExtractor

llm = OpenAI(temperature=0)
compressor = LLMChainExtractor.from_llm(llm)
compression_retriever = ContextualCompressionRetriever(base_compressor=compressor, base_retriever=retriever)

compressed_docs = compression_retriever.get_relevant_documents("What did the president say about Ketanji Jackson Brown")
print(compressed_docs[0].page_content)

Now, let's talk team players. The EnsembleRetriever brings different algorithms together for a grand performance. In this example, BM25 and FAISS Retrievers join forces:

from langchain.retrievers import BM25Retriever, EnsembleRetriever
from langchain.vectorstores import FAISS

bm25_retriever = BM25Retriever.from_texts(doc_list).set_k(2)
faiss_vectorstore = FAISS.from_texts(doc_list, OpenAIEmbeddings())
faiss_retriever = faiss_vectorstore.as_retriever(search_kwargs={"k": 2})

ensemble_retriever = EnsembleRetriever(
    retrievers=[bm25_retriever, faiss_retriever], weights=[0.5, 0.5]
)

docs = ensemble_retriever.get_relevant_documents("apples")
print(docs[0].page_content)

Now, here's something for those who want more from a document. The MultiVector Retriever lets you query with multiple vectors per document. Here's how you can split documents into smaller chunks:

from langchain.retrievers.multi_vector import MultiVectorRetriever
from langchain.vectorstores import Chroma
from langchain.embeddings import OpenAIEmbeddings
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.storage import InMemoryStore
from langchain.document_loaders import TextLoader
import uuid

loaders = [TextLoader("file1.txt"), TextLoader("file2.txt")]
docs = [doc for loader in loaders for doc in loader.load()]
text_splitter = RecursiveCharacterTextSplitter(chunk_size=10000)
docs = text_splitter.split_documents(docs)

vectorstore = Chroma(collection_name="full_documents", embedding_function=OpenAIEmbeddings())
store = InMemoryStore()
id_key = "doc_id"
retriever = MultiVectorRetriever(vectorstore=vectorstore, docstore=store, id_key=id_key)

doc_ids = [str(uuid.uuid4()) for _ in docs]
child_text_splitter = RecursiveCharacterTextSplitter(chunk_size=400)
sub_docs = [sub_doc for doc in docs for sub_doc in child_text_splitter.split_documents([doc])]
for sub_doc in sub_docs:
    sub_doc.metadata[id_key] = doc_ids[sub_docs.index(sub_doc)]

retriever.vectorstore.add_documents(sub_docs)
retriever.docstore.mset(list(zip(doc_ids, docs)))

Lastly, for those who want a balance between accuracy and context, there's the Parent Document Retriever:

from langchain.retrievers import ParentDocumentRetriever

loaders = [TextLoader("file1.txt"), TextLoader("file2.txt")]
docs = [doc for loader in loaders for doc in loader.load()]

child_splitter = RecursiveCharacterTextSplitter(chunk_size=400)
vectorstore = Chroma(collection_name="full_documents", embedding_function=OpenAIEmbeddings())
store = InMemoryStore()
retriever = ParentDocumentRetriever(vectorstore=vectorstore, docstore=store, child_splitter=child_splitter)

retriever.add_documents(docs, ids=None)

retrieved_docs = retriever.get_relevant_documents("query")

These retrievers make LangChain a powerhouse for retrieving information. Whether you want focused content, multiple perspectives, or a balanced approach, there's a retriever for you. And hey, don't forget the documentation for more explorations!

A self-querying retriever constructs structured queries from natural language inputs and applies them to its underlying VectorStore. Its implementation is shown in the following code:

from langchain.chat_models from ChatOpenAI
from langchain.chains.query_constructor.base from AttributeInfo
from langchain.retrievers.self_query.base from SelfQueryRetriever

metadata_field_info = [AttributeInfo(name="genre", description="...", type="string"), ...]
document_content_description = "Brief summary of a movie"
llm = ChatOpenAI(temperature=0)

retriever = SelfQueryRetriever.from_llm(llm, vectorstore, document_content_description, metadata_field_info)

retrieved_docs = retriever.invoke("query")

The WebResearchRetriever performs web research based on a given query -

from langchain.retrievers.web_research import WebResearchRetriever

# Initialize components
llm = ChatOpenAI(temperature=0)
search = GoogleSearchAPIWrapper()
vectorstore = Chroma(embedding_function=OpenAIEmbeddings())

# Instantiate WebResearchRetriever
web_research_retriever = WebResearchRetriever.from_llm(vectorstore=vectorstore, llm=llm, search=search)

# Retrieve documents
docs = web_research_retriever.get_relevant_documents("query")

For our examples, we can also use the standard retriever already implemented as part of our vector store object as follows -

We can now query the retrievers. The output of our query will be document objects relevant to the query. These will be ultimately utilized to create relevant responses in further sections.

Next Chapter: Lanchain's 3rd Module: Agents

Top 5 High Demand Programming Languages in 2024

Jaydeep Biswas — Fri, 05 Jan 2024 18:29:20 +0000

Will Python Maintain Its Dominance?
Hey fellow coders! As we step into the exciting realm of 2024, it's only natural to wonder which programming languages are holding their ground and, more importantly, where you should focus your up-skilling efforts. With the ever-expanding universe of coding languages, it's crucial to stay ahead of the curve. So, let's dive into the top five programming languages that are set to dominate in 2024 and beyond.

1. Python: The Unstoppable Force

Python has been on a relentless climb up the programming language charts, and in 2024, it's showing no signs of slowing down. Praised for its versatility and rapid development capabilities, Python has become a go-to language for various applications. Statista ranks it as the third most-used language in 2023, and the TIOBE Index currently crowns it as number one. Whether you're into AI, automation, or workflow optimization, strong Python skills are a hot commodity in the job market.

2. Java: The Time-Tested Titan

Java, born in 1995, has stood the test of time and remains a stalwart in the programming world. A recent survey of 14 million developer jobs placed Java as the third most in-demand language. Widely used in web development, cloud computing, IoT applications, and large-scale enterprise tools, Java offers a sense of job security that's hard to match.

3. Kotlin: Powering the Future of App Development

If you're into Android or cross-platform app development, Kotlin should be on your radar. Endorsed by Google as the official language for Android development in 2017, Kotlin has steadily gained popularity. Fintech enthusiasts, take note – SumUp is in search of a senior backend Kotlin engineer to contribute to an innovative in-app point-of-sale solution in the heart of Paris.

4. JavaScript: The Ever-Adaptable Dynamo

JavaScript, with its incredible adaptability, continues to hold a prime spot among the most in-demand languages. Powering over 98% of all websites in some capacity, JavaScript is the force behind the dynamic and interactive content you encounter on your devices daily. From laptops to smartphones to smart TVs, JavaScript is the unsung hero of front-end web development.

5. Rust: Releasing Performance and Safety

Enter Rust, a programming language designed for speed, safety, and practicality. Positioned as a systems programming language, Rust runs like a speeding bullet, preventing segfaults and guaranteeing thread safety. Its memory efficiency makes it ideal for embedded systems, making it a rising star in the programming constellation.

Rust Benefits:

Fast and efficient for high-performance applications.
Safe and reliable, perfect for mission-critical software.
Easy to learn with a supportive developer community.

Rust Cons:

Limited support/documentation as compared to other languages.
Challenges in integration with existing codebases. As we embark on this coding journey into 2024, these five languages stand out as the champions of the programming arena. Whether you're a Python aficionado, a Java enthusiast, a Kotlin connoisseur, a JavaScript wizard, or a Rust explorer, there's a world of opportunities awaiting your skills. Happy coding, and may your 2024 be filled with endless lines of success! 🚀✨

LangChain's 1st Module: Model I/O 🦜🤖

Jaydeep Biswas — Wed, 03 Jan 2024 18:29:56 +0000

In the last post we have gone through the Installation and Setup of LangChain.

In the realm of LangChain, the pivotal element shaping any application lies in the language model. This module lays the foundation for effective interaction with language models, ensuring a seamless integration process. 🚀

Key Components of Model I/O 🧩

LLMs and Chat Models (used interchangeably): 🗣️

LLMs:

Definition: Pure text completion models.
Input/Output: Take a text string as input and return a text string as output.

Chat Models:

Definition: Models leveraging a language model as a base, differing in input and output formats.
Input/Output: Accept a list of chat messages as input and return a Chat Message.

Prompts: 📝

Templatize, dynamically select, and manage model inputs. This enables the creation of flexible and context-specific prompts guiding the language model's responses.

Output Parsers: 📤

These components extract and format information from model outputs. They prove invaluable for converting raw language model output into structured data or specific formats required by the application.

LLMs: 🧠

LangChain's integration with Large Language Models (LLMs), such as OpenAI, Cohere, and Hugging Face, constitutes a fundamental aspect of its functionality. LangChain itself doesn't host LLMs but provides a uniform interface for interacting with various LLMs.

This section outlines the usage of the OpenAI LLM wrapper in LangChain, applicable to other LLM types. Assuming it's installed, let's initialize the LLM:

from langchain.llms import OpenAI
llm = OpenAI()

LLMs adhere to the Runnable interface, the basic building block of the LangChain Expression Language (LCEL). This implies support for invoke, ainvoke, stream, astream, batch, abatch, astream_log calls.

LLMs accept strings as inputs or objects coerced to string prompts, including List[BaseMessage] and PromptValue. Now, let's delve into some examples:

response = llm.invoke("List the seven wonders of the world.")
print(response)

You can alternatively call the stream method to stream the text response.

for chunk in llm.stream("Where were the 2012 Olympics held?"):
    print(chunk, end="", flush=True)

Chat Models: Revolutionizing Conversations in LangChain 💬

In the dynamic realm of LangChain, the integration of chat models emerges as a pivotal force, breathing life into interactive chat applications. These models, a specialized variant of language models, wield the power of internal language models while showcasing a distinctive interface tailored around chat messages as both inputs and outputs. Let's embark on an in-depth exploration of leveraging OpenAI's chat model within the LangChain ecosystem.

from langchain.chat_models import ChatOpenAI
chat = ChatOpenAI()

Within the language of LangChain, chat models seamlessly interact with various message types, including AIMessage, HumanMessage, SystemMessage, FunctionMessage, and ChatMessage (sporting an arbitrary role parameter). The stalwarts among these are undeniably HumanMessage, AIMessage, and SystemMessage.

from langchain.schema.messages import HumanMessage, SystemMessage
messages = [
    SystemMessage(content="You are Michael Jordan."),
    HumanMessage(content="Which shoe manufacturer are you associated with?"),
]
response = chat.invoke(messages)
print(response.content)

Unveiling the Power of Prompts 🎭

Prompts, the architects of coherent and relevant language model outputs, assume a central role in the LangChain narrative. From straightforward instructions to intricate few-shot examples, handling prompts within LangChain is a streamlined journey, all thanks to a suite of dedicated classes and functions.

Crafting a Dynamic Prompt with `PromptTemplate` 🖋️

from langchain.prompts import PromptTemplate

# Simple prompt with placeholders
prompt_template = PromptTemplate.from_template(
    "Tell me a {adjective} joke about {content}."
)

# Filling placeholders to create a prompt
filled_prompt = prompt_template.format(adjective="funny", content="robots")
print(filled_prompt)

For chat models, where prompts evolve into more structured conversations with messages assigned specific roles, LangChain introduces the ChatPromptTemplate.

Shaping an Interactive Chat Prompt 🤔

from langchain.prompts import ChatPromptTemplate

# Defining a chat prompt with various roles
chat_template = ChatPromptTemplate.from_messages(
    [
        ("system", "You are a helpful AI bot. Your name is {name}."),
        ("human", "Hello, how are you doing?"),
        ("ai", "I'm doing well, thanks!"),
        ("human", "{user_input}"),
    ]
)

# Formatting the chat prompt
formatted_messages = chat_template.format_messages(name="Bob", user_input="What is your name?")
for message in formatted_messages:
    print(message)

This strategic approach empowers the creation of chatbots that are not only interactive but also dynamic in their responses, adapting to the nuances of the conversation.

Both PromptTemplate and ChatPromptTemplate seamlessly integrate with the LangChain Expression Language (LCEL), positioning themselves as integral components within more extensive and intricate workflows—a topic we'll delve deeper into shortly.

Custom prompt templates become the artisans' tools, essential for tasks demanding unique formatting or specific instructions. The artistry involves defining input variables and crafting a custom formatting method, providing LangChain the flexibility to cater to a diverse array of application-specific needs.

Discover the power of few-shot prompting in LangChain, a feature that empowers models to learn from examples. This proves indispensable for tasks requiring contextual understanding or recognition of specific patterns. Few-shot prompt templates can be meticulously constructed from a set of examples or with the aid of an Example Selector object—unveil more on this here. Embrace the journey of transforming prompts into dialogues, where language models breathe life into interactive narratives within the LangChain universe.

The Power of Output Parsers in LangChain 🛠️

Output parsers stand as the unsung heroes in the vibrant ecosystem of LangChain, playing a pivotal role in shaping the responses generated by language models. This section is an exploration of the nuanced world of output parsers, accompanied by code examples utilizing LangChain's diverse set, including PydanticOutputParser, SimpleJsonOutputParser, CommaSeparatedListOutputParser, DatetimeOutputParser, and XMLOutputParser.

PydanticOutputParser: Crafted Precision ✨

LangChain introduces the PydanticOutputParser, a gem for parsing responses into Pydantic data structures. Let's delve into a step-by-step example to witness its prowess:

# Initializing the language model
model = OpenAI(model_name="text-davinci-003", temperature=0.0)

# Defining the desired data structure using Pydantic
class Joke(BaseModel):
    setup: str = Field(description="question to set up a joke")
    punchline: str = Field(description="answer to resolve the joke")

    @validator("setup")
    def question_ends

_with_question_mark(cls, field):
        if field[-1] != "?":
            raise ValueError("Badly formed question!")
        return field

# Setting up a PydanticOutputParser
parser = PydanticOutputParser(pydantic_object=Joke)

# Creating a prompt with format instructions
prompt = PromptTemplate(
    template="Answer the user query.\n{format_instructions}\n{query}\n",
    input_variables=["query"],
    partial_variables={"format_instructions": parser.get_format_instructions()},
)

# Defining a query to prompt the language model
query = "Tell me a joke."

# Combining prompt, model, and parser for structured output
prompt_and_model = prompt | model
output = prompt_and_model.invoke({"query": query})

# Parsing the output using the parser
parsed_result = parser.invoke(output)

# The result is a structured object
print(parsed_result)

SimpleJsonOutputParser: Decoding JSON-Like Elegance 🌐

When dealing with JSON-like outputs, LangChain's SimpleJsonOutputParser takes the stage. Here's a glimpse into its functionality:

# Creating a JSON prompt
json_prompt = PromptTemplate.from_template(
    "Return a JSON object with `birthdate` and `birthplace` key that answers the following question: {question}"
)

# Initializing the JSON parser
json_parser = SimpleJsonOutputParser()

# Crafting a chain with the prompt, model, and parser
json_chain = json_prompt | model | json_parser

# Streaming through the results
result_list = list(json_chain.stream({"question": "When and where was Elon Musk born?"}))

# The result is a list of JSON-like dictionaries
print(result_list)

CommaSeparatedListOutputParser: Unraveling Lists with Ease 📜

The CommaSeparatedListOutputParser steps in when extracting comma-separated lists from model responses becomes imperative. Witness its simplicity in action:

# Initializing the parser
output_parser = CommaSeparatedListOutputParser()

# Creating format instructions
format_instructions = output_parser.get_format_instructions()

# Creating a prompt to request a list
prompt = PromptTemplate(
    template="List five {subject}.\n{format_instructions}",
    input_variables=["subject"],
    partial_variables={"format_instructions": format_instructions}
)

# Defining a query to prompt the model
query = "English Premier League Teams"

# Generating the output
output = model(prompt.format(subject=query))

# Parsing the output using the parser
parsed_result = output_parser.parse(output)

# The result is a list of items
print(parsed_result)

DatetimeOutputParser: Unveiling Temporal Insights 🕰️

LangChain's DatetimeOutputParser is tailored for parsing datetime information. Experience its capabilities firsthand:

# Initializing the DatetimeOutputParser
output_parser = DatetimeOutputParser()

# Creating a prompt with format instructions
template = """
Answer the user's question:
{question}
{format_instructions}
"""

prompt = PromptTemplate.from_template(
    template,
    partial_variables={"format_instructions": output_parser.get_format_instructions()},
)

# Creating a chain with the prompt and language model
chain = LLMChain(prompt=prompt, llm=OpenAI())

# Defining a query to prompt the model
query = "when did Neil Armstrong land on the moon in terms of GMT?"

# Running the chain
output = chain.run(query)

# Parsing the output using the datetime parser
parsed_result = output_parser.parse(output)

# The result is a datetime object
print(parsed_result)

These examples unfold the versatility of LangChain's output parsers, adept at structuring diverse model responses to cater to various applications and formats. Output parsers emerge as indispensable tools, elevating the usability and interpretability of language model outputs within the LangChain ecosystem.

Next Chapter : LangChain's 2nd Module: Retrieval

Discover Infinite Possibilities with LangChain 🦜🚀

Jaydeep Biswas — Tue, 02 Jan 2024 18:28:08 +0000

Introduction 🌐

Embarking on the exploration of LangChain feels like diving into a world of limitless possibilities, where innovation meets the power of language models. At its core, LangChain serves as a revolutionary framework, meticulously crafted to empower developers in the creation of applications that seamlessly tap into the capabilities of language models. This toolkit goes beyond mere coding; it represents a holistic ecosystem, streamlining the process from conceptualization to implementation through its diverse components.

LangChain Libraries 📚

As I delve into the intricacies of LangChain, I encounter its foundational elements – the Python and JavaScript-based LangChain Libraries. These libraries, the backbone of LangChain, provide the necessary interfaces and integrations for various components. They not only offer a runtime for combining these components into coherent chains but also present ready-made implementations for immediate application.

LangChain Templates 🎨

Adding to the richness of the LangChain experience are the purpose-built LangChain Templates. These deployable reference architectures cater to a spectrum of tasks, offering a solid starting point for projects, whether it be a conversational chatbot or a sophisticated analytical tool.

LangServe: Transforming Projects into Web Services 🌐

Enter LangServe, a versatile library designed for deploying LangChain chains as REST APIs. This tool is the linchpin for transforming LangChain projects into accessible and scalable web services, a crucial step in taking applications to the next level.

LangSmith: A Developer's Playground 🛠️

Complementing this toolkit is LangSmith, a dedicated developer platform that serves as a testing ground for debugging, evaluating, and monitoring chains built on any large language model (LLM) framework. Its seamless integration with LangChain makes it an indispensable companion for developers striving to refine and perfect their applications.

Synergy in Action 🔄

The synergy among these components empowers me to navigate the development, productionization, and deployment of applications with unparalleled ease. With LangChain, the journey begins by crafting applications using the libraries, drawing inspiration from templates for guidance. LangSmith steps in to aid in inspecting, testing, and monitoring chains, ensuring a continuous enhancement process. Finally, the deployment phase becomes a seamless experience with LangServe, effortlessly transforming any chain into an API.

Conclusion 🌟

As I prepare to delve deeper into the intricacies of setting up LangChain, the excitement builds. The prospect of creating intelligent, language model-powered applications is within reach, thanks to the comprehensive and user-friendly ecosystem that LangChain provides.

Getting Started with LangChain: Installation and Setup 🚀

Dive into the world of LangChain with a straightforward installation process. Follow this step-by-step guide to set up LangChain seamlessly.

Installation Steps 🛠️

Installing LangChain

Install LangChain using either pip or conda with the following command:

pip install langchain

For those who prefer the latest features and are comfortable with a bit more adventure, you can install LangChain directly from the source. Clone the repository and navigate to the langchain/libs/langchain directory. Then, run:

pip install -e .

For experimental features, consider installing langchain-experimental:

pip install langchain-experimental

LangChain CLI 📟

The LangChain CLI is a helpful tool for LangChain templates and LangServe projects. Install it with:

pip install langchain-cli

LangServe Setup 🌐

LangServe is essential for deploying LangChain chains as a REST API and gets installed alongside the LangChain CLI.

Integrating External Tools (OpenAI Example) 🤝

LangChain often requires integrations with external entities. For example, for OpenAI's model APIs, install the OpenAI Python package:

pip install openai

To access the API, set your OpenAI API key as an environment variable:

export OPENAI_API_KEY="your_api_key"

Alternatively, pass the key directly in your Python environment:

import os
os.environ['OPENAI_API_KEY'] = 'your_api_key'

LangChain Modules 🧩

LangChain offers modular components, including:

Model I/O: Facilitates interaction with language models.
Retrieval: Enables access to application-specific data.
Agents: Empower applications to select tools based on high-level directives.
Chains: Pre-defined, reusable compositions for application development.
Memory: Maintains application state across multiple chain executions.

LangChain Expression Language (LCEL) 💬

LangChain Expression Language (LCEL) is a declarative way to compose modules. Example LCEL snippet:

from langchain.chat_models import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
from langchain.schema import BaseOutputParser

# Example chain
chain = ChatPromptTemplate() | ChatOpenAI() | CustomOutputParser()

Next Steps 🚀

Now that the basics are covered, delve deeper into each LangChain module, learn the LangChain Expression Language, explore common use cases, implement them, deploy end-to-end applications with LangServe, and leverage LangSmith for debugging, testing, and monitoring.

Unleash the full potential of LangChain as you craft powerful language model applications!
Next Chapter: LangChain's 1st Module: Model I/O

Understanding LangChain: Unveiling the Power Behind the Platform 🦜🔗

Jaydeep Biswas — Mon, 01 Jan 2024 18:29:52 +0000

Introduction 🌐

A few months ago, I stumbled upon LangChain while immersed in a Language Model (LLM) project. Since then, I have been captivated by its capabilities, though I often find that it is not as widely comprehended as it deserves to be. This is unsurprising, considering that when I initially explored LangChain, its utility wasn't immediately apparent. However, as I delved deeper and incorporated it into my projects, the true potential of LangChain gradually unfolded.

Unraveling LangChain's Power 🔍

Overview
LangChain, at its core, is a versatile platform that tackles a spectrum of challenges encountered in language-related projects. It goes beyond the surface-level understanding and requires a hands-on approach to fully appreciate its capabilities.

Use Cases
LangChain finds its strength in solving intricate problems associated with language processing and understanding. Here's an exploration of the key areas where LangChain excels:

1. Dynamic Language Modeling 🌐
LangChain shines in the realm of Language Model projects. Its ability to adapt dynamically to varying linguistic contexts sets it apart. This adaptability is especially beneficial when dealing with diverse datasets and complex linguistic structures.

2. Problematic Misunderstandings Clarified 🤔
When discussing LangChain with others, a common theme emerges - a lack of clear comprehension. LangChain addresses this by acting as a bridge, clarifying misunderstandings and facilitating a more intuitive grasp of its functionalities.

3. Project-Driven Revelation 🚀
The true power of LangChain unfolds progressively as you integrate it into your projects. It is not merely a tool; it's an evolving solution that reveals its depth and sophistication as you navigate through different use cases.

Elaborating on LangChain's Role 🛠️

LangChain's role becomes more apparent when you explore its features in-depth:

- Adaptive Learning Mechanism 🔄
LangChain boasts an adaptive learning mechanism, making it well-suited for projects that involve continuous learning and evolving linguistic patterns. This adaptability ensures that the platform remains effective across diverse language landscapes.

- Semantic Clarity Engine 🧠
The platform acts as a Semantic Clarity Engine, dissecting complex linguistic constructs and providing a clearer understanding. This is particularly valuable in scenarios where the meaning behind language nuances can significantly impact project outcomes.

Overview 🌐

LangChain emerges as a crucial abstraction in the landscape of Large Language Models (LLMs). In the era post-ChatGPT, where various LLMs abound, LangChain provides a unified interface to seamlessly experiment with and switch between models, offering both performance and cost advantages.

Prompt Engineering 🤖

Crafting the right question, or "prompt," is pivotal for meaningful results with LLMs. LangChain facilitates prompt engineering through prompt templates, enabling the integration of external data from enterprise databases. The platform also introduces the intriguing concept of chaining LLMs, allowing the output of one question to serve as input for another.

Retrieval Augmented Generation (RAG) 🔄

LangChain delves into the importance of context in LLMs, introducing Retrieval Augmented Generation (RAG). By augmenting retrieval with context, users can significantly enhance the relevance and accuracy of responses. LangChain provides tools for embedding private data into a vector store, allowing for intelligent context provision to LLMs based on the user's data.

Tools and Agents 🧰

The platform introduces the "ReAct" prompting technique, encouraging LLMs to think step by step for enhanced reasoning. LangChain defines tools as wrappers around APIs and offers various pre-built tools. These tools can be associated with agents, which are then employed to augment LLM capabilities. This approach is particularly useful in overcoming the limitation of frozen-in-time knowledge in LLMs.

Response Processing and Conversational Interfaces 🗣️

LangChain addresses the challenge of processing LLM output by providing features like Callbacks, Streaming, and Batching. This ensures efficient data processing, whether for building conversational applications or APIs.

Conclusion 🎉

In essence, LangChain is a versatile Swiss Army Knife for LLMs, offering a comprehensive suite of tools and features. This overview only scratches the surface of its capabilities, encouraging users to explore further and unlock the full potential of language models.

I will continue to write blogs and tutorials on Langchain. To learn more and engage the community, here is a link to their documentation. LangChain Documentation 📘

DEV Community: Jaydeep Biswas

Seeking advice on optimizing response time and handling multiple requests on AWS instance with NVIDIA A10G GPU

LangChain's 3rd Module: Agents🦜🕴️

Exploring LangChain's Agents 🔍🤖

Tools 🛠️

Examples of Tools 📚🔧

DuckDuckGo

DataForSeo

Shell (bash)

Return to Agents ↩️🤖

Memory in Agents 🧠🤖

Agent Types 🤖📝

Prebuilt Agents 🤖🛠️

LangChain Gmail Toolkit 📧🔧

SQL Database Agent 📊🤖

Pandas DataFrame Agent 🐼📊🤖

Jira Toolkit 📅🔧

LangChain's 2nd Module: Retrieval🦜🐕

Document Loaders 📄

Text File Loader

CSV Loader

PDF Loaders

PyPDFLoader (Basic PDF Parsing)

MathPixLoader (Mathematical Content and Diagrams)

PyMuPDFLoader (Fast PDF Parsing with Detailed Metadata)

PDFMiner Loader (Granular Control over Text Extraction)

AmazonTextractPDFParser utilizes AWS Textract for OCR and other advanced PDF parsing features.

PDFMinerPDFasHTMLLoader generates HTML from PDF for semantic parsing.

PDFPlumberLoader provides detailed metadata and supports one document per page.

Example I - Slack 💬

Example II - Figma 🎨

Document Transformers 🔄

Text Embedding Model 📝➡️🔠

Vector Stores 🗄️

Retrievers 🔍

Top 5 High Demand Programming Languages in 2024

LangChain's 1st Module: Model I/O 🦜🤖

Key Components of Model I/O 🧩

LLMs and Chat Models (used interchangeably): 🗣️

Prompts: 📝

Output Parsers: 📤

LLMs: 🧠

Chat Models: Revolutionizing Conversations in LangChain 💬

Unveiling the Power of Prompts 🎭

Crafting a Dynamic Prompt with PromptTemplate 🖋️

Shaping an Interactive Chat Prompt 🤔

The Power of Output Parsers in LangChain 🛠️

PydanticOutputParser: Crafted Precision ✨

SimpleJsonOutputParser: Decoding JSON-Like Elegance 🌐

CommaSeparatedListOutputParser: Unraveling Lists with Ease 📜

DatetimeOutputParser: Unveiling Temporal Insights 🕰️

Discover Infinite Possibilities with LangChain 🦜🚀

Introduction 🌐

LangChain Libraries 📚

LangChain Templates 🎨

LangServe: Transforming Projects into Web Services 🌐

LangSmith: A Developer's Playground 🛠️

Synergy in Action 🔄

Conclusion 🌟

Getting Started with LangChain: Installation and Setup 🚀

Installation Steps 🛠️

Installing LangChain

LangChain CLI 📟

LangServe Setup 🌐

Integrating External Tools (OpenAI Example) 🤝

LangChain Modules 🧩

LangChain Expression Language (LCEL) 💬

Next Steps 🚀

Understanding LangChain: Unveiling the Power Behind the Platform 🦜🔗

Introduction 🌐

Unraveling LangChain's Power 🔍

Elaborating on LangChain's Role 🛠️

Overview 🌐

Prompt Engineering 🤖

Retrieval Augmented Generation (RAG) 🔄

Tools and Agents 🧰

Response Processing and Conversational Interfaces 🗣️

Conclusion 🎉

Crafting a Dynamic Prompt with `PromptTemplate` 🖋️