LangChain Explained: A Practical Guide For Beginners

In this guide, I will explain what LangChain is, why it exists, and how it simplifies the development of LLM-powered applications. Instead of presenting isolated concepts, we will build our understanding step by step, starting with a simple LLM integration and gradually exploring prompt templates, output parsers, memory, Retrieval-Augmented Generation (RAG), and tool calling.

By the end of this article, you will understand how these building blocks fit together and how LangChain orchestrates them into a cohesive workflow.

About Me

I am David Archanjo, a Full-Stack Engineer with 10+ years of experience in the technology industry. I am passionate about software development and AI Engineering, and I enjoy sharing practical knowledge through technical articles that help developers build real-world applications.

---

Contents

• Prerequisites
• What Is LangChain?
• Why Learn and Use LangChain?
• The Core Concept of LangChain
• Hello, World!: LLM Integration
• Prompt Templates
• Output Parsers
• LangChain Expression Language (LCEL)
• Conversational Memory Management
• Retrieval-Augmented Generation (RAG)
• Tool Calling and Agents
• Next Steps
• Wrap Up

---

Prerequisites

To get the most out of this guide, you should have:

• Python 3.10 or later and pip installed.
• Basic knowledge of Python.
• A text editor or IDE such as Visual Studio Code or PyCharm.
• A basic understanding of Large Language Models (LLMs).

---

What Is LangChain?

LangChain is a toolkit that helps developers build smart AI applications powered by language models.

Think of a language model as a very smart brain, but on its own, it:

• Cannot access our private files
• Cannot search the web
• Does not remember past conversations
• Cannot use tools like calculators or databases

LangChain connects that brain to tools, data, and memory.

---

Why Learn and Use LangChain?

To understand why we need LangChain, we first have to understand the limitation of LLMs like GPT-4 or Claude.

While LLMs are great, they are like "brains in a jar". They are powerful at processing information but have no internet access, no memory of past conversations (beyond a single session), and no way to interact with our specific private data, just to name a few of their limitations.

LangChain is the "Swiss Army Knife" that connects LLMs to the rest of the world.

Five Reasons Why LangChain is Necessary

1. "Chaining" Multiple Steps

A simple prompt is easy, but complex tasks (like writing a research paper) require a sequence.

• Without LangChain: We have to manually take the output of one prompt, clean it up, and paste it into the next.
• With LangChain: We create a "Chain." For example: Step 1: Search Google -> Step 2: Summarize results -> Step 3: Translate to Portuguese. LangChain takes all of these pieces and runs as a flow.

2. Context Management (Memory)

LLMs are technically "stateless". Once a session ends, they don't remember who we are, what we talked about.

• Without LangChain: Developers have to manually store chat logs in a database and feed them back to the AI every time, which is problematic and time-consuming.
• With LangChain: It provides built-in "Memory" modules. It automatically tracks the conversation history and feeds the relevant parts back to the LLM so it remembers what we said five minutes ago.

3. Data Awareness (RAG)

LLMs are trained on public data up to a certain date. They don't know about our private stuff, like emails, spreadsheets, or today’s news.

• The Solution: How LangChain helps:
• Without LangChain: We must feed LLMs manually with additional data in such a way they are eventually able to reason based on what was provide and then answer ours questions or perform the given action accordingly.
• With LangChain: It provides "Document Loaders" and "Vector Store" integrations, allowing LLMs to "read" our specific documents and answer questions based only on that data. That's Retrieval Augmented Generation (RAG) to the rescue.

4. Model-agnostic Approach

The AI field moves fast. Today OpenAI's model is the leader; tomorrow it might Anthropic's Claude or an open-source model like Gemma 4.

Without LangChain: If we want or need to switch models, we have to rewrite our entire codebase or application because every AI company has a different API.
With LangChain: It acts as an abstraction layer. We write our code once using LangChain’s syntax, and we can swap the underlying LLM with just one line of code.

5. Using "Agents"

This is the most advanced part of LangChain. An Agent is an AI that can decide for itself which tool to use.

• Without LangChain: We must manually code the logic that decides which tool to use and when. For example, if a user asks, "What is Apple’s stock price and how does it affect my investements?", we would need to:
  • Detect that stock data is required
  • Call a stock price API
  • Parse the response
  • Retrieve the user’s portfolio data
  • Run calculations
  • Format the final answer
  • All decision-making and orchestration logic have to be hard-coded
• With LangChain: We define tools (e.g. Stock API, Calculator, Database), and the Agent automatically decides which tool to use and in what order. The LLM reasons through the task, selects the appropriate tools, executes them, and combines the results into a final response. All this without we manually scripting every decision step.

---

The Core Concept of LangChain

At its core, LangChain is built around one simple idea: everything is a composable building block.

Just like LEGO bricks, each component in LangChain does one job and is designed to connect with other components. We stack them together to build something bigger and more powerful.

These building blocks are:

Building Block	What It Does
LLM / Chat Model	The brain. Receives input and generates a response.
Prompt Template	Structures and formats the input sent to the LLM.
Output Parser	Transforms the LLM's raw text response into structured data.
Memory	Stores and retrieves conversation history across interactions.
Retriever / Vector Store	Fetches relevant documents from our private data.
Tools	External capabilities the LLM can call (APIs, calculators, databases).
Agent	The decision-maker that chooses which tools to use and when.

How They Connect: The Chain

The term "LangChain" comes from the idea of chaining these blocks together into a pipeline. A typical flow looks like this:

  User Input
      ↓
Prompt Template (formats the input)
      ↓
     LLM        (generates a response)
      ↓
 Output Parser  (structures the response)
      ↓
 Final Output

As the application grows in complexity, more blocks are added to the chain:

  User Input
      ↓
   Memory       (adds conversation history)
      ↓
  Retriever     (fetches relevant documents)
      ↓
Prompt Template (formats everything together)
      ↓
     LLM        (reasons and generates a response)
      ↓
 Output Parser  (structures the response)
      ↓
 Final Output

In the section that follow, we will explore each of these building blocks individually, starting from the simplest possible example and progressively adding more components until we are able to get a complete AI application powered by LangChain.

---

Hello, World!: LLM Integration

As with any new technology, the best place to start is with a basic "Hello, World!". In LangChain, the most fundamental building block is LLM integration: sending a prompt to a language model and receiving its response.

So let's get started!

Installation

First, we install LangChain and the OpenAI integration packages:

pip install langchain langchain-openai

Each package serves a specific purpose:

Package	Purpose
langchain	Provides the core LangChain framework, including abstractions for prompts, document loaders, text splitters, embeddings, vector stores, retrievers, and chains.
langchain-openai	Adds integration with OpenAI-compatible models, allowing LangChain to interact with chat models through the ChatOpenAI class.

Then, set our OpenAI API key as an environment variable:

export OPENAI_API_KEY="sk-xxxxx"

Our First LLM Call

from langchain_openai import ChatOpenAI

# Initialize the model
llm = ChatOpenAI(model="gpt-4o")

# Invoke the model with a simple prompt
response = llm.invoke("What is LangChain in one sentence?")

print(response.content)

Output:

LangChain is a framework that helps developers build applications powered by large language models by connecting them to tools, data, and memory.

Breaking Down The Code

1. We instantiated a ChatOpenAI model, pointing to gpt-4o.
2. We called llm.invoke() with a plain string message as argument.
3. LangChain handled the API communication and returned an AIMessage object.
4. We printed out response.content to display the response in plain text.

Running LangChain with a Local OpenAI-Compatible API

Although the code examples in this article use the ChatOpenAI class, they do not require the official OpenAI API. One of the major advantages of LangChain is that its OpenAI integration can communicate with any inference server implementing the OpenAI API specification. This allows us to switch between cloud providers and locally hosted models with little or no code changes.

For this article, we can use a locally hosted language model exposed through an OpenAI-compatible API using, for instance, llama.cpp. This setup allows every example to run entirely on our machine while keeping the code identical to what we would write when using the official OpenAI service.

I personally use llamafile, a standalone distribution of llama.cpp, to serve open models locally, like the Gemma 4:

./llamafile \
  -m gemma-4-E2B-it-Q4_K_M.gguf \
  --server \
  --host 0.0.0.0 \
  --port 8000 \
  --alias gpt-4o

Here, --alias gpt-4o tells the server to expose the local model under the name gpt-4o. From LangChain's perspective, it behaves exactly like the OpenAI model with the same name, allowing every example in this article to work with the same ChatOpenAI initialization:

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o")

without requiring any code changes.

To make this work transparently, we MUST configure the following environment variables:

export OPENAI_API_KEY="sk-xxxxx"
export OPENAI_BASE_URL="http://localhost:8000/v1"

The OPENAI_BASE_URL environment variable overrides the default OpenAI endpoint, causing every request made through ChatOpenAI to be sent to the local inference server instead of OpenAI's servers.

Although the local server does not validate API keys, the OpenAI client library still expects the OPENAI_API_KEY environment variable to exist. Since the key is never verified by llama.cpp, any non-empty value is sufficient.

Using a local OpenAI-compatible API offers several benefits while learning LangChain:

• No OpenAI subscription required.
• No API usage costs.
• Complete privacy, since prompts and responses remain on our machine.
• Offline development, provided the model has already been downloaded.
• The same LangChain code works with both local and cloud-hosted models.

If you already have an OpenAI API key, simply remove the OPENAI_BASE_URL environment variable and replace the placeholder value of OPENAI_API_KEY with your own key. The examples throughout this article will continue to work without modification.

In addition to llama.cpp, ChatOpenAI can communicate with any inference server exposing an OpenAI-compatible API, which includes other tools such as LM Studio, vLLM, Ollama and SGLang.

Switching to a Different Model

Because LangChain acts as an abstraction layer, swapping the underlying model requires changing just one line. The following shows how would be using Anthropic's Claude:

# !pip install langchain-anthropic
from langchain_anthropic import ChatAnthropic

llm = ChatAnthropic(model="claude-sonnet-4-6")

response = llm.invoke("What is LangChain in one sentence?")
print(response.content)

The rest of our application code stays exactly the same. This is the power of LangChain's model-agnostic approach.

---

Prompt Templates

Hardcoding a message directly into .invoke() works for simple cases, but real applications need dynamic, reusable prompts, i.e. prompts that change based on user input or context.

That's when Prompt Templates comes to rescue!

What is a Prompt Template?

A Prompt Template is a reusable prompt with placeholders that get filled in at runtime. Think of it like a form letter where we define the structure once and fill in the specific details later.

Without Prompt Template

# Manually formatting strings every time is messy and error-prone
user_language = "Portuguese"
user_topic = "LangChain"
prompt = f"Explain {user_topic} in simple terms. Answer in {user_language}."

response = llm.invoke(prompt)

print(response.content)

This version works, but it becomes unmanageable as prompts grow more complex over time.

With Prompt Template

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

# Define the prompt structure once
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful technical writer who explains concepts simply."),
    ("human", "Explain {topic} in simple terms. Answer in {language}.")
])

# Initialize the model
llm = ChatOpenAI(model="gpt-4o")

# Create a chain that combines the prompt template and the model
chain = prompt | llm

# Invoke with dynamic values
response = chain.invoke({"topic": "LangChain", "language": "Portuguese"})

print(response.content)

Key Benefits of Prompt Templates

• Reusability: Define the prompt once, use it many times with different inputs.
• Separation of Concerns: The prompt structure is separate from the application logic.
• Role-based Messaging: Easily define system, human, and ai roles in a conversation.
• Composability: Templates connect directly to LLMs using the | pipe operator, forming a chain.

---

Output Parsers

By default, LangChain returns LLM's response as an AIMessage object. While this works well for conversational applications, many real-world systems require structured data that can be processed programmatically.

For example, an application may need to:

• store the response in a database;
• serialize it as JSON;
• populate a web interface;
• call another service;
• execute business rules based on individual fields.

Working with plain text quickly becomes inconvenient.

That's when Output Parsers comes to rescue!

The Problem: Default Response Output

Suppose we are building an AI assistant that recommends learning resources.

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o")

response = llm.invoke("Recommend a resource for learning LangChain.")

print(type(response))   # <class 'langchain_core.messages.ai.AIMessage'>
print(response.content) # I recommend starting with the official LangChain documentation...

The response is human-readable, but extracting fields such as the resource name or URL would require additional parsing.

Parsing to Plain Text

When the application only needs the generated text, StrOutputParser removes the surrounding AIMessage object and returns a simple string.

from langchain_core.output_parsers import StrOutputParser

parser = StrOutputParser()

chain = llm | parser

response = chain.invoke("Explain what a Vector Database is.")

print(type(response)) # <class 'langchain_core.messages.base.TextAccessor'>
print(response)       # A vector database is a specialized database...

This is the simplest parser and the one we will probably use most often in chatbot applications.

NOTE: langchain_core.messages.base.TextAccessor is a specialized, string-like object in LangChain created to ensure backward compatibility during the framework's transition from method-based to property-based text access for messages.

Parsing to Structured Data (JSON)

For more complex use cases, we can instruct the LLM to return structured data and parse it automatically:

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import JsonOutputParser

# Define the prompt structure once
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant. Always respond in valid JSON."),
    ("human", "{question}")
])

# Initialize the model
llm = ChatOpenAI(model="gpt-4o")

# Initialize the output parser
parser = JsonOutputParser()

# Create a chain that combines the prompt template, the model, and the output parser
chain = prompt | llm | parser

# Invoke with a dynamic question
response = chain.invoke({
    "question": "Give me three benefits of LangChain as a JSON array with 'title' and 'description' fields. Items in the JSON array must be sorted alphabetically by the 'title' field."
})

# Now the response is a Python list we can iterate over
for benefit in response:
    print(f"- {benefit['title']}: {benefit['description']}")

Output:

- Composability: LangChain allows you to chain components together into powerful pipelines.
- Ecosystem: A rich library of integrations for tools, memory, and data retrieval.
- Model Agnostic: Switch between LLM providers with a single line of code.

Producing Structured Objects

Now imagine we are developing an learning platform.

Instead of free-form text, we would like the LLM to recommend a course using a predefined schema.

from pydantic import BaseModel, Field

# Define a structured output model for course recommendations
class CourseRecommendation(BaseModel):
    course: str = Field(description="Course name")
    level: str = Field(description="Difficulty level")
    duration_hours: int = Field(description="Estimated duration")
    reason: str = Field(description="Why the course is recommended")

LangChain offers two ways to obtain this object.

Option 1: Native Structured Output

Modern language models can generate structured data directly.

LangChain exposes this capability through with_structured_output(), allowing the model to return a validated Pydantic object without requiring an explicit parser.

# Initialize the model
llm = ChatOpenAI(model="gpt-4o")

# Configure the LLM to use the structured output model
llm = llm.with_structured_output(CourseRecommendation, method="json_schema")

# Invoke the model with a prompt
recommendation = llm.invoke("Recommend a beginner course for learning LangChain.")

# The response is now a structured object we can access its fields directly
print(f"Course: {recommendation.course}")
print(f"Level: {recommendation.level}")
print(f"Duration: {recommendation.duration_hours} hours")
print(f"Reason: {recommendation.reason}")

Output:

Course: LangChain for Beginners
Level: Beginner
Duration: 80 hours
Reason: Hands-on implementation of core concepts, focusing on prompt engineering, chaining, and retrieval mechanisms.

Notice that recommendation is already a CourseRecommendation instance — no manual JSON parsing required. This is the preferred approach whenever the underlying model supports structured outputs.

Option 2: Using PydanticOutputParser

Not every model supports native structured output.

In these situations, LangChain can instruct the model to produce JSON matching a predefined schema and then automatically convert it into a Python object.

from langchain_core.output_parsers import PydanticOutputParser
from langchain_core.prompts import ChatPromptTemplate

# Initialize the Pydantic parser for the structured output model
parser = PydanticOutputParser(pydantic_object=CourseRecommendation)

# Define the prompt structure with format instructions for the Pydantic parser
prompt = ChatPromptTemplate.from_messages([
    (
        "system",
        """
You are an AI education advisor.

Return ONLY valid JSON.

{format_instructions}
"""
    ),
    (
        "human",
        "Recommend a {level} course for learning {topic}."
    )
])

# Create a chain that combines the prompt template, the model, and the Pydantic output parser
chain = prompt | llm | parser

# Invoke with dynamic values and format instructions
recommendation = chain.invoke({"topic": "LangChain", "level": "beginner", "format_instructions": parser.get_format_instructions()})

# The response is a structured object we can access its fields directly
print(f"Course: {recommendation.course}")
print(f"Level: {recommendation.level}")
print(f"Duration: {recommendation.duration_hours} hours")
print(f"Reason: {recommendation.reason}")

PydanticOutputParser validates the generated JSON and converts it into a CourseRecommendation object before returning it the final result.

Which Approach Should Be Used?

Approach	Recommended Use
StrOutputParser	When the application only needs plain text.
with_structured_output()	When the model supports native structured outputs. Preferred approach.
PydanticOutputParser	When working with models that don't support native structured outputs or when explicit formatting instructions are required.

---

LangChain Expression Language (LCEL)

In the previous sections, we observed the ǀ pipe operator appearing in the code examples, right?

chain = prompt | llm | parser

This is LangChain Expression Language (LCEL), which is the standard, unified way to compose LangChain components into pipelines, and it is worth understanding deeply because it underpins everything in LangChain.

What is LCEL?

LCEL is a declarative syntax for building chains. The ǀ operator works like a Unix pipe: the output of the component on the left becomes the input of the component on the right.

This single line chain = prompt | llm | parser is read as: "Take the prompt, pipe it into the LLM, then pipe the result into the parser". This piping instruction replaces what used to require many lines of manual orchestration code:

# Step 1: Format the prompt
prompt = prompt.invoke({"topic": "LangChain"})

# Step 2: Invoke the LLM
response = llm.invoke(prompt)

# Step 3: Parse the output
result = parser.invoke(response)

Why LCEL is Better?

The pipe operator isn't just syntactic sugar. It creates a Runnable pipeline.

Instead of this:

prompt = prompt.format(topic="LangChain")
response = llm.invoke(prompt)
result = parser.invoke(response)

we simply write:

chain = prompt | llm | parser

result = chain.invoke({"topic": "LangChain"})

The Runnable Interface

LCEL works because every LangChain component (prompts, LLMs, parsers, retrievers, tools) implements the same interface called Runnable. A Runnable is any object that exposes these standard methods:

Method	Description
.invoke(input)	Runs the chain with a single input and returns a single output.
.stream(input)	Runs the chain and streams the output token by token.
.batch(inputs)	Runs the chain on a list of inputs in parallel.
.ainvoke(input)	Async version of .invoke() for non-blocking execution.
.astream(input)	Async version of .stream().
.abatch(inputs)	Async version of .batch().

You find more information about the Runnable interface here.

Because every component shares this interface, they can be freely composed with the ǀ operator without any adapter code.

Streaming with LCEL

One of LCEL's most practical benefits is first-class streaming support. Instead of waiting for the full response, we can stream tokens as they are generated:

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

# Initialize the model
llm = ChatOpenAI(model="gpt-4o")

# Initialize the output parser
parser = StrOutputParser()

# Initialize the prompt template to be used with dynamic questions
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant."),
    ("human", "{question}")
])

# Create the chain combining the prompt template, the model, and the output parser
chain = prompt | llm | parser

# Stream the response token by token
for chunk in chain.stream({"question": "Explain LangChain in 3 sentences."}):
    print(chunk, end="", flush=True)

This gives us a responsive, real-time experience with no additional configuration required.

Batch Processing with LCEL

Need to process multiple inputs at once? Use .batch():

responses = chain.batch([
    {"question": "What is LangChain?"},
    {"question": "What is a Prompt Template?"},
    {"question": "What is RAG?"}
])

for response in responses:
    print(response)
    print("---")

LangChain automatically runs these in parallel, making batch processing significantly faster than sequential calls.

Branching with RunnableParallel

LCEL also supports parallel branches, which consist of running multiple chains simultaneously and merging their results:

from langchain_core.runnables import RunnableParallel
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

llm = ChatOpenAI(model="gpt-4o")
parser = StrOutputParser()

# Set two separate chains
summary_chain = (
    ChatPromptTemplate.from_template("Summarize this topic in one sentence: {topic}")
    | llm
    | parser
)

example_chain = (
    ChatPromptTemplate.from_template("Give one practical example of: {topic}")
    | llm
    | parser
)

# Run both chains in parallel and combine the results
chain = RunnableParallel(
    summary=summary_chain,
    example=example_chain
)

result = chain.invoke({"topic": "LangChain"})

print("Summary:", result["summary"])
print("Example:", result["example"])

Output:

Summary: LangChain is a framework for building LLM-powered applications by composing modular components.
Example: A customer support chatbot that retrieves answers from a company's private knowledge base using RAG.

---

Conversational Memory Management

So far, our chain treats every invocation as an isolated interaction. Send a message, get a response, and it's done. But real AI-powered conversational applications need to remember what was said before.

That's when Conversational Memory Management comes to rescue!

The Problem without Memory

chain.invoke({"question": "Hey, David here from Brazil."})
# Response: "Nice to meet you, David!"

chain.invoke({"question": "What's my home country?"})
# Response: "I don't know which country you are." ← The LLM has no clue :)

Adding Memory with Conversation History

LangChain provides memory through chat history management, which in our example is leveraged by InMemoryChatMessageHistory. The most practical approach is using RunnableWithMessageHistory, which automatically injects past messages into every new invocation:

from langchain_openai import ChatOpenAI
from langchain_core.chat_history import InMemoryChatMessageHistory
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.runnables.history import RunnableWithMessageHistory

# Store for session histories
session_store = {}

# Define the function to retrieve or create a session history
def get_session_history(session_id: str):
    if session_id not in session_store:
        session_store[session_id] = InMemoryChatMessageHistory()
    return session_store[session_id]

# Prompt now includes a placeholder for chat history
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant. Remember the conversation context."),
    MessagesPlaceholder(variable_name="chat_history"),
    ("human", "{question}")
])

llm = ChatOpenAI(model="gpt-4o")
parser = StrOutputParser()

chain = prompt | llm | parser

# Wrap the chain with memory management
chain = RunnableWithMessageHistory(
    chain,
    get_session_history,
    input_messages_key="question",
    history_messages_key="chat_history"
)

# Session config to identify the conversation
config = {"configurable": {"session_id": "user_david_session"}}

# First message
response1 = chain.invoke(
    {"question": "Hey, David here from Brazil."},
    config=config # type: ignore
)
print(response1)  # "Nice to meet you, David!"

# Second message: now the LLM remembers
response2 = chain.invoke(
    {"question": "What's my home country?"},
    config=config # type: ignore
)
print(response2)  # "Based on what you told me, your home country is Brazil!"

Note: InMemoryChatMessageHistory stores conversations only in memory, so all chat history is lost when the application exits. For long-lived AI assistants, LangChain provides persistent implementations that store conversation history in external databases (PostgresChatMessageHistory, RedisChatMessageHistory) or storage services (DynamoDBChatMessageHistory, S3ChatMessageHistory), allowing users to resume previous conversations across application restarts.

Breaking Down The Code

1. The user sends a new message.
2. LangChain fetches the chat history for the given session_id.
3. The history is injected into the prompt via the MessagesPlaceholder.
4. The LLM receives the full conversation context and responds accordingly.
5. The new message and response are saved back to the history automatically.

---

Retrieval-Augmented Generation (RAG)

Even with memory, our LLM only knows what it was trained on. It has no access to our private documents, internal knowledge bases, or real-time data. RAG solves this by fetching relevant information and injecting it into the prompt before the LLM responds.

How RAG Works

 User Question
      ↓
  Retriever      (searches our documents for relevant chunks)
      ↓
Relevant Chunks  (injected into the prompt as context)
      ↓
Prompt Template  (combines question + context)
      ↓
     LLM         (answers based on the retrieved context)
      ↓
 Final Answer

RAG Implementation

Unlike chat models, RAG applications also require an embedding model capable of converting text into vector embeddings.

For this guide, I run locally a dedicated nomic-embed-text model on port 8001, exposing the standard OpenAI Embeddings API:

./llamafile \
  -m nomic-embed-text-v1.5.f16.gguf \
  --embedding \
  --server \
  --host 0.0.0.0 \
  --port 8001 \
  --alias nomic-embed-text \
  --rope-scaling yarn \
  --rope-freq-scale 0.75 \
  -c 8192 \
  -b 8192 \
  --ubatch-size 8192

This separation is a common production architecture, where specialized models are used for different tasks and runs in dedicated environment.

Step 1: Install dependencies

RAG requires two additional dependencies to be installed besides the packages introduced earlier in this guide (i.e. langchain and langchain-openai):

pip install langchain-community langchain-chroma

Chroma is an open-source vector database used to store embeddings and perform similarity searches over our documents

Step 2: Load and split a document

The first step is loading the document(s) that will serve as the application's knowledge base. Since LLMs have a limited context window, storing an entire document as a single block of text is inefficient and usually ineffective.

Instead, we split the document(s) into smaller overlapping chunks. During retrieval, the vector database can search these chunks individually and return only the most relevant pieces for the user's question.

from langchain_community.document_loaders import TextLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter

# Load the target document
loader = TextLoader("company_credentials.txt")
documents = loader.load()

# Split into smaller chunks for better retrieval
text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)
chunks = text_splitter.split_documents(documents)

Step 3: Create a Vector Store

The next step is converting every text chunk into a numerical representation called embedding.

Embeddings capture the semantic meaning of the text, allowing similar pieces of information to be found even when different words are used.

Once the embeddings are generated, they are stored in a Chroma vector database so we can perform similarity searches over them later.

from langchain_chroma import Chroma
from langchain_openai import OpenAIEmbeddings

# Initialize the embedding model
embeddings = OpenAIEmbeddings(
    model="nomic-embed-text",
    base_url="http://localhost:8001/v1",
    check_embedding_ctx_length=False
)

# Generate embeddings and index them in Chroma (in-memory)
vector_store = Chroma.from_documents(documents=chunks, embedding=embeddings)

# Create a retriever
retriever = vector_store.as_retriever(search_kwargs={"k": 3})

The retriever will now be responsible for searching the vector database and returning the most relevant document chunks whenever a user asks a question.

Step 4: Build the RAG Chain

With the retriever in place, we can finally build our RAG pipeline.

When a user submits a question, LangChain first sends it to the retriever, which searches the vector store for the most relevant chunks. Those retrieved chunks are then injected into the prompt as contextual information before the prompt is sent to the language model.

Finally, the LLM generates an answer based on both the user's question and the retrieved context.

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough

prompt = ChatPromptTemplate.from_messages([
    ("system", """You are a helpful assistant. Answer the user's question based only on the context provided below. If the answer cannot be found in the context, say that you don't know.

     Context: {context}
     """),
    ("human", "{question}")
])

llm = ChatOpenAI(model="gpt-4o")
parser = StrOutputParser()

# Set up the RAG chain
chain = (
    {"context": retriever, "question": RunnablePassthrough()}
    | prompt
    | llm
    | parser
)

# Ask a question with information only found in the ingested document
response = chain.invoke("What are my login credentials for Jira?")
print(response)

Notice that the retriever is automatically executed as part of the chain. The retrieved document chunks populate the {context} placeholder, while the original user question is passed unchanged to {question} through RunnablePassthrough().

The internal RAG workflow:

Company Handbook
       ↓
 Text Splitter
       ↓
     Chunks
       ↓
   Embeddings
       ↓
Chroma Vector Store
       ↑ (similarity search)
 User Question
       ↓
Retrieved Chunks
       ↓
Prompt Template
       ↓
      LLM
       ↓
  Final Output

Why RAG Matters?

Without RAG	With RAG
LLM answers from training data only	LLM answers from our private documents
Knowledge is frozen at training cutoff	Knowledge can be updated anytime
May hallucinate missing information	Grounds answers in real retrieved content
One-size-fits-all responses	Responses tailored to our specific data

---

Tool Calling and Agents

The final and most powerful building block is Tool Calling. This is what transforms a passive LLM into an active Agent, i.e. one that can take actions in the real world.

What is Tool Calling?

Tool Calling allows the LLM to:

• Recognize that it needs external information or capability to answer a question.
• Select the right tool from a set of available tools.
• Call the tool with the correct arguments.
• Use the tool's response to formulate the final answer.

Defining a Tool

In LangChain, any Python function can become a tool using the @tool decorator.

Imagine we are building a minimal personal AI assistant with two capabilities: telling the current time and identifying the operating system it's running on.

import platform
from datetime import datetime
from langchain_core.tools import tool

@tool
def get_current_time() -> str:
    """Return the current local date and time."""
    return datetime.now().strftime("%Y-%m-%d %H:%M:%S")

@tool
def get_operating_system() -> str:
    """Returns the name and architecture of the host operating system.

    Use this tool whenever the user asks about their system environment, 
    OS name, computer platform, or machine architecture.
    """
    os_name = platform.system()
    architecture = platform.machine()

    return f"Operating System: {os_name}, Architecture: {architecture}"

Although these tools are simple, they illustrate an important concept: the LLM does not execute business logic itself. Instead, it decides when a tool should be called and what arguments should be passed.

Building an Agent with Tools

Now let's give the LLM access to these tools.

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain.agents import create_tool_calling_agent, AgentExecutor

tools = [get_current_time, get_operating_system]

# Initialize the model
llm = ChatOpenAI(model="gpt-4o")

# Define the prompt structure
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant. Use tools whenever they are useful."),
    ("human", "{input}"),
    ("placeholder", "{agent_scratchpad}"),
])

# Build the agent and executor
agent = create_tool_calling_agent(llm=llm, tools=tools, prompt=prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

# Example usage
input = "What time is it right now?"
response = agent_executor.invoke({"input": input})
print(f"{input}:", response["output"])

input = "What is today's date?"
response = agent_executor.invoke({"input": input})
print(f"{input}:", response["output"])

input = "What platform is the user running on?"
response = agent_executor.invoke({"input": input})
print(f"{input}:", response["output"])

Output (with verbose=True, the execution flow becomes visible):

> Entering new AgentExecutor chain...

Invoking: `get_current_time` with `{}`


2026-07-14 21:51:01The current time is 21:51:01 on July 14, 2026.

> Finished chain.
What time is it right now?: The current time is 21:51:01 on July 14, 2026.


> Entering new AgentExecutor chain...

Invoking: `get_current_time` with `{}`


2026-07-14 21:51:03The current date is July 14, 2026.

> Finished chain.
What is today's date?: The current date is July 14, 2026.


> Entering new AgentExecutor chain...

Invoking: `get_operating_system` with `{}`


Operating System: Linux, Architecture: x86_64The user is running on **Linux** with an **x86_64** architecture.

> Finished chain.
What platform is the user running on?: The user is running on **Linux** with an **x86_64** architecture.

Notice that the LLM never executes the Python functions directly. It only determines which tool should be called and what arguments should be passed. LangChain is responsible for executing the functions and returning their results to the model.

The Agent Loop

What makes agents powerful is their reasoning loop:

         User Request
              ↓
      +----------------+
      |      LLM       |
      +----------------+
              │
      "Do I need a tool?"
      Yes ↓       ↓ No
Select a Tool   Final Output
          ↓
  Execute Tool
          ↓
  Tool Returns Result
          ↓
      +----------------+
      |      LLM       |
      +----------------+
              │
    "Do I need another tool?"

This loop continues until the LLM determines it has enough information to answer the user's question.

---

Next Steps

Now that we understand the foundational building blocks of LangChain, here are some natural next steps:

• LangGraph: A framework built on top of LangChain for building stateful, multi-agent workflows as graphs.
• LangSmith: LangChain's observability platform for debugging and monitoring our chains and agents in production.
• LangChain Templates: Pre-built application templates to accelerate development.
• LangChain Integrations Hub: Explore the hundreds of available integrations for document loaders, vector stores, tools, and LLM providers.

LangChain is a rapidly evolving ecosystem. The best way to master it is to build something real. Start small, add one building block at a time, and let complexity grow naturally as our application demands it.

---

Wrap Up

Throughout this guide, we explored LangChain from the ground up, building our understanding one component at a time.

Here is a summary of everything we covered:

Component	Purpose	Key Takeaway
LLM Integration	Connect to a language model	One unified API for all LLM providers
Prompt Templates	Structure and reuse prompts	Separate prompt logic from application logic
Output Parsers	Structure the LLM's response	Convert raw text into usable data formats
LCEL	Compose components into pipelines	The ǀ operator connects everything together
Memory	Persist conversation history	Give the LLM context across multiple turns
RAG	Connect the LLM to private data	Ground responses in our own documents
Tool Calling / Agents	Give the LLM the ability to act	Let the LLM decide which tools to use and when

All the Python examples presented in this guide are available in this accompanying GitHub repository, making it easy to run, modify, and experiment with each example on your own.

Happy AI coding 🚀