Getting started with LLM and LangChain

Large Language Models (LLMs) have revolutionized the field of Natural Language Processing (NLP) and Artificial Intelligence (AI). However, building and training LLMs can be a complex and resource-intensive process, requiring significant computational power and expertise. That's where langchain comes in—LangChain provides a range of tools and features that address various complexities with building applications using LLMs, such as prompt crafting, response structuring, and model integration.

In this blog post, we will go over some 101-level information on LLMs, and then walk through a demo on how to build a very simple LLM-based application to query for information. Let's dive into the world of Large Language Models and explore the ways in which LangChain is making them accessible and affordable for everyone.

First, what are Large Language Models?

Large language models (LLMs) are a class of artificial intelligence (AI) algorithms that have been trained on massive amounts of text data, for example, text from books, articles, websites, or other written content. LLMs use this data to learn how to predict the next word or words that will occur in a sequence. Given some initial words in a sentence or paragraph, an LLM can generate a natural-sounding continuation of the text.

There are several different types of LLMs, each with its own unique architecture and training methodologies. Some of the most popular types of LLMs include:

• Recurrent Neural Networks (RNNs): These neural networks are designed to handle sequential data, meaning language models with context, such as long sentences or paragraphs.

• Transformers: Transformers were developed in 2017 by OpenAI models and have since become among the most popular LLM designs. Transformers are trained using a self-supervised learning learning method that enables them to analyze and encode vast amounts of text data while still maintaining the context of the information.

• Generative Pre-trained Transformer 3 (GPT-3): This is one of the most advanced LLMs currently available, with 175 billion parameters as of 2021. GPT-3 is a neural network-based language model that is incredibly powerful at generating and understanding human language.

Large language models have many applications, including natural language search engines, text-to-speech systems, chatbots and automated customer service, and language translation software. As LLMs continue to improve, they are becoming increasingly useful tools for businesses in a wide range of industries.

LLMs are already starting to change the way we interact with written content, and as their capabilities continue to grow, businesses and other organizations are starting to realize the benefits of using these systems to streamline their operations. For those interested in AI’s future, large language models are an exciting area to follow, and LangChain is a great platform to learn more and build new AI skills.

What is LangChain?

LangChain offers a comprehensive framework for creating applications that leverage the power of large language models. With LangChain, users are able to develop chatbots, personal assistants, and other AI-powered tools that can perform a range of tasks such as summarization, analysis, Q&A generation, code comprehension, API interaction, and more.

The LangChain framework comes equipped with pre-designed chains for efficiently executing complex tasks. Users can customize existing chains or build entirely new ones by utilizing the vast library of components that LangChain offers.

The LangChain framework has numerous components that can be used to develop custom applications. These components include:

Prompts: Prompt templates play a crucial role in streamlining user input to ensure effective communication with a natural language model. Its primary function is to format user input in a fashion that matches the expectations of the language model. These templates are crucial in providing context and specifying inputs to help the model perform the intended task efficiently. As an illustration, a chatbot prompt template may incorporate the user's name alongside their question to enhance contextual relevance and improve the performance of the language model.

Chains: Chains are pre-designed language model components that accomplish specific tasks. These are pre-designed sets of components that can process input and generate output. Chains can be used as-is or customized to meet the specific requirements of a project. Custom chains can be designed by combining multiple components, creating powerful and flexible applications.

Agents: Agents are responsible for language model training and communication. Agents train models by determining the optimal input for the model and optimizing its output. They are also responsible for receiving user input, processing it, and providing appropriate output from the model. The LangChain Agent handles different input/output formats, language models, and APIs.

LLMs: Language models (LLMs) are at the core of LangChain's functionality. They analyze, summarize, generate Q&A, and understand code. LangChain enables users to create their own models or use pre-trained models from the LangChain library. The flexibility of LangChain allows users to train models on any type of data or use pre-trained models for any type of use case.

Callbacks: Callbacks enable developers to enhance the functionality of the LangChain components by adding customized algorithms. The callbacks can be used to modify the output generated by the language model or modify the input before input pre-processing.

Memory: Memory allows users to store data across multiple sessions, enabling LangChain to provide a more personalized experience to the users. Memory can be used to store user preferences, chat history, and interaction patterns, allowing LangChain to learn from previous interactions and adapt to user's needs.

Output Parsers: Output parsers take responsibility for organizing the response output that Language Model (LLM) generates. The parser performs some functions, such as the elimination of unwanted or irrelevant content, addition of supplementary information, or alteration of response format. As a significant component of the LangChain framework, output parsers play an essential role in maintaining outcome relevance, accuracy, and structure. It ensures that the LLM's responses are more decipherable, thus improving ease of interpretation and increases in practicality of such responses.

One of the most significant benefits that LangChain offers is the ability to access LLMs models without expending high development costs or needing extensive technical expertise. LangChain’s suite of tools deliver valuable assistance for prompt crafting, response structuring, model integration and other programming-related requirements.

Now that we’re familiar with the concepts, let’s walk through a simple example to better understand how various components work together. We’ll build a service that checks whether Wikipedia contains cat recordings (who doesn’t love a cat video?):

Installation and Setup

Let’s first configure the environment. Assuming Python is already installed in your working environment, the next step is to install the LangChain library using pip:

pip install langchain

As we are going to use the OpenAI’s language models, we will install OpenAI SDK as well:

pip install openai

Next, as a critical step in connecting to OpenAI's API, it is necessary to set the OPENAI_API_KEY as an environment variable. First, sign in to your OpenAI account and navigate to account settings. Then, head over to the View API Keys option and generate a secret key, which you will need to copy. Within your Python script, use the os module to access the dictionary of available environment variables and set the "OPENAI_API_KEY" to the secret API key you copied earlier.

import os
os.environ["OPENAI_API_KEY"] = "your-api-key-here"

Get predictions
Import the LLM wrapper:

from langchain.llms import OpenAI

We can then initialize the wrapper with any arguments.

gen_llm = OpenAI(openai_api_key=”…”, verbose=False)

Initialize the APIChain to query from the API endpoint:

from langchain.chains.api import open_meteo_docs

chain_new = APIChain.from_llm_and_api_docs(gen_llm, “https://cat-fact.herokuapp.com/facts/", verbose=True)

Now we’re ready to ask a question to the LLM:

chain_new.run(‘Does wikipedia have a cat recording?’)

You should get an output like this:

> Entering new APIChain chain…
https://cat-fact.herokuapp.com/facts/


[{“status”:{“verified”:true,”sentCount”:1},”_id”:”58e008800aac31001185ed07",”user”:”58e007480aac31001185ecef”,”text”:”Wikipedia has a recording of a cat meowing, because why not?”,”__v”:0,”source”:”user”,”updatedAt”:”2020–08–23T20:20:01.611Z”,”type”:”cat”,”createdAt”:”2018–03–06T21:20:03.505Z”,”deleted”:false,”used”:false},{“status”:{“verified”:true,”sentCount”:1},”_id”:”58e008630aac31001185ed01",”user”:”58e007480aac31001185ecef”,”text”:”When cats grimace, they are usually \”taste-scenting.\” They have an extra organ that, with some breathing control, allows the cats to taste-sense the air.”,”__v”:0,”source”:”user”,”updatedAt”:”2020–08–23T20:20:01.611Z”,”type”:”cat”,”createdAt”:”2018–02–07T21:20:02.903Z”,”deleted”:false,”used”:false},{“status”:{“verified”:true,”sentCount”:1},”_id”:”58e00a090aac31001185ed16",”user”:”58e007480aac31001185ecef”,”text”:”Cats make more than 100 different sounds whereas dogs make around 10.”,”__v”:0,”source”:”user”,”updatedAt”:”2020–08–23T20:20:01.611Z”,”type”:”cat”,”createdAt”:”2018–02–11T21:20:03.745Z”,”deleted”:false,”used”:false},{“status”:{“verified”:true,”sentCount”:1},”_id”:”58e009390aac31001185ed10",”user”:”58e007480aac31001185ecef”,”text”:”Most cats are lactose intolerant, and milk can cause painful stomach cramps and diarrhea. It’s best to forego the milk and just give your cat the standard: clean, cool drinking water.”,”__v”:0,”source”:”user”,”updatedAt”:”2020–08–23T20:20:01.611Z”,”type”:”cat”,”createdAt”:”2018–03–04T21:20:02.979Z”,”deleted”:false,”used”:false},{“status”:{“verified”:true,”sentCount”:1},”_id”:”58e008780aac31001185ed05",”user”:”58e007480aac31001185ecef”,”text”:”Owning a cat can reduce the risk of stroke and heart attack by a third.”,”__v”:0,”source”:”user”,”updatedAt”:”2020–08–23T20:20:01.611Z”,”type”:”cat”,”createdAt”:”2018–03–29T20:20:03.844Z”,”deleted”:false,”used”:false}]

> Finished chain.

Out[4]:
‘ Yes, Wikipedia has a recording of a cat meowing.’

Congratulations! Your basic setup of an LLM-based application using LangChain is now working successfully.