LlamaIndex vs LangChain: Agent and RAG Orchestration

LlamaIndex vs LangChain: Which Should You Use for Agent and RAG Orchestration?

TL;DR: LlamaIndex and LangChain are MIT-licensed Python frameworks for building LLM applications. LlamaIndex is strongest at the data side of RAG: parsing, indexing, retrieval and query engines. LangChain is strongest at agent orchestration: its create_agent runs on LangGraph, which adds built-in checkpoints and human approval. Choose LlamaIndex for retrieval-heavy apps and LangChain for durable, tool-heavy agents.

This comparison covers the two frameworks as they stand in November 2026: llama-index 0.14 with llama-index-workflows 2.x, and langchain 1.4 with langgraph 1.2. For multi-agent frameworks such as CrewAI and AutoGen, see the CrewAI vs LangGraph vs AutoGen comparison.

Key Takeaways

• LlamaIndex treats your data as the centre of the application. Its core abstractions are documents, nodes, indexes, retrievers and query engines, and an agent usually sits on top of them.
• LangChain treats the agent loop as the centre. Its core abstraction in version 1.x is create_agent, a model calling tools in a loop, shaped by middleware and executed on the LangGraph runtime.
• For RAG, LlamaIndex ships more retrieval machinery out of the box: node parsers, ingestion pipelines, routers, response synthesizers, rerankers and a PropertyGraphIndex. LangChain gives you the building blocks (loaders, splitters, vector stores, retrievers) and expects you to assemble them.
• For agent orchestration, both are event- or graph-driven. LlamaIndex has event-driven Workflows with @step functions and AgentWorkflow handoffs; LangChain has create_agent plus LangGraph's state graphs.
• Durability is the sharpest difference. LangGraph saves state through a checkpointer after every step. LlamaIndex documents that Workflows have no built-in checkpointer: you snapshot Context yourself or use the DBOS integration.
• The two combine cleanly. A common production pattern is LlamaIndex for ingestion and retrieval, exposed as a tool to a LangChain or LangGraph agent.

What are LlamaIndex and LangChain?

Both started in late 2022 as Python libraries for connecting large language models to external data and tools. They have since grown in different directions, and that history explains most of the trade-offs below.

LlamaIndex (originally GPT Index) is an MIT-licensed framework maintained by LlamaIndex, Inc. It began as a way to index private documents so an LLM could answer questions about them, and retrieval-augmented generation (RAG) is still its centre of gravity. You load data into Documents, split them into Nodes, build an Index (most often a VectorStoreIndex), and query it through a Retriever or a Query Engine that also synthesises the answer. Agents arrived later and are built on Workflows, an event-driven orchestration layer now published as its own package, llama-index-workflows. The company also sells hosted document services such as LlamaParse for parsing complex PDFs, which the open-source framework can call but does not require.

LangChain is an MIT-licensed framework from LangChain, Inc. It began as a library of chains and integrations, and with the 1.0 release it was refocused around agents. The main entry point is now create_agent from langchain.agents, which the documentation describes as a model calling tools in a loop, wrapped in a harness of prompt, tools and middleware. That agent runs on LangGraph, the company's lower-level orchestration runtime, so it inherits LangGraph's state, persistence and interrupts. Retrieval is still there (document loaders, text splitters, embeddings, vector stores and retrievers), but it is one capability among many rather than the organising idea.

A short way to remember them: LlamaIndex is a data framework that learned to run agents, and LangChain is an agent framework that also does retrieval.

How do LlamaIndex and LangChain compare at a glance?

Dimension	LlamaIndex	LangChain
Centre of the design	Data: indexes, retrievers, query engines	Agent loop: model, tools, middleware
Current versions (Nov 2026)	llama-index 0.14, llama-index-workflows 2.x	langchain 1.4, langgraph 1.2
RAG primitives	Documents, Nodes, IngestionPipeline, VectorStoreIndex, PropertyGraphIndex, query engines, response synthesizers	Loaders, text splitters, embeddings, vector stores, retrievers
Agent entry point	FunctionAgent, ReActAgent, CodeActAgent	create_agent
Orchestration model	Event-driven Workflows (@step, typed events)	LangGraph state graph (nodes, edges, shared state)
Multi-agent	AgentWorkflow with can_handoff_to, orchestrator agents, custom Workflows	Subagents as tools, LangGraph supervisors and handoffs, Deep Agents
Durable execution	Snapshot Context.to_dict() yourself, or DBOS integration	Built-in checkpointers (in-memory, SQLite, Postgres)
Human-in-the-loop	InputRequiredEvent / HumanResponseEvent	HumanInTheLoopMiddleware, LangGraph interrupt()
Languages	Python, TypeScript (LlamaIndex.TS)	Python, JavaScript/TypeScript
Commercial add-ons	LlamaParse and hosted document services	LangSmith tracing, evaluation and deployment
License	MIT	MIT
Best fit	Document-heavy RAG and knowledge assistants	Tool-heavy agents with approvals and long-running state

Which is better for RAG: LlamaIndex or LangChain?

For a retrieval-heavy application, LlamaIndex usually gets you further with less code, because more of the RAG pipeline is a named, configurable component instead of something you write.

LlamaIndex's RAG stack is layered. An IngestionPipeline runs a list of transformations (node parsers, metadata extractors, embedding models) over your documents and can cache results so re-ingesting a changed corpus only processes what changed. Indexes then decide how those nodes are stored and looked up. On the query side, a query engine combines a retriever, optional node postprocessors (similarity cut-offs, rerankers) and a response synthesizer with selectable response modes. A RouterQueryEngine picks between several query engines based on their descriptions, and a PropertyGraphIndex builds and queries a knowledge graph when vector similarity alone misses relationships. Each of these is replaceable without rewriting the others.

The minimum version is four lines:

from llama_index.core import SimpleDirectoryReader, VectorStoreIndex

documents = SimpleDirectoryReader("./policies").load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine(similarity_top_k=4)

print(query_engine.query("What is the refund window for annual plans?"))

LangChain's RAG stack is a kit of parts. You choose a loader, split with something like RecursiveCharacterTextSplitter, embed into one of many vector store integrations, and get a retriever from the store. The LangChain documentation describes three architectures: 2-step RAG, where retrieval always runs before generation (predictable latency and cost); agentic RAG, where the agent decides when and how to retrieve during reasoning (more flexible, variable latency); and hybrid, which adds validation steps between the two. LangChain's own emphasis in 2026 is on the agentic form, with retrieval exposed as a tool:

from langchain.agents import create_agent
from langchain.tools import tool
from langchain_community.document_loaders import DirectoryLoader
from langchain_core.vectorstores import InMemoryVectorStore
from langchain_openai import OpenAIEmbeddings
from langchain_text_splitters import RecursiveCharacterTextSplitter

docs = DirectoryLoader("./policies").load()
chunks = RecursiveCharacterTextSplitter(chunk_size=800, chunk_overlap=100).split_documents(docs)
store = InMemoryVectorStore(OpenAIEmbeddings(model="text-embedding-3-small"))
store.add_documents(chunks)

@tool
def search_policies(query: str) -> str:
    """Search internal policy documents and return matching passages."""
    hits = store.similarity_search(query, k=4)
    return "\n\n".join(f"[{d.metadata.get('source')}]\n{d.page_content}" for d in hits)

agent = create_agent(
    model="openai:gpt-4.1",
    tools=[search_policies],
    system_prompt="Answer only from search_policies results and cite the source file.",
)
result = agent.invoke({"messages": [{"role": "user", "content": "What is the refund window for annual plans?"}]})
print(result["messages"][-1].content)

The LangChain version is longer, but every step is visible and you own it. That is an advantage when your retrieval logic is unusual (custom scoring, permission filtering, several stores) and a cost when it is not.

Where the difference really shows is messy source data. Tables, scanned PDFs, nested headings and slide decks are where most RAG quality is lost, and LlamaIndex has invested heavily in that layer, both in its open-source node parsers and in the commercial LlamaParse service. LangChain's loaders get the text out; LlamaIndex gives you more control over how it is structured before it is embedded. If your corpus is clean Markdown, the gap is small. If it is a decade of PDFs, test LlamaIndex's ingestion first.

Which is better for agent orchestration: LlamaIndex or LangChain?

For agents that call many tools, run for a long time or need approval before consequential actions, LangChain (with LangGraph underneath) is the stronger default in 2026. LlamaIndex agents are capable, but more of the production plumbing is left to you.

LlamaIndex agents are Workflows. A FunctionAgent is a pre-built workflow for models with native tool calling; ReActAgent and CodeActAgent use different prompting strategies for models or tasks where that helps. For your own control flow, you subclass Workflow and write @step methods that receive one typed event and emit another. The runtime routes events to whichever step accepts them, so branching, loops and parallel fan-out fall out of the event types. Shared state lives in a Context object.

from llama_index.core.agent.workflow import FunctionAgent
from llama_index.core.tools import QueryEngineTool

policy_tool = QueryEngineTool.from_defaults(
    query_engine,
    name="policy_docs",
    description="Answers questions about refund and billing policies.",
)
agent = FunctionAgent(
    tools=[policy_tool],
    llm=llm,
    system_prompt="Use policy_docs for any policy question and quote it.",
)
response = await agent.run(user_msg="Can a customer get a refund after 45 days?")

Notice that the tool here is a whole query engine, with its retriever, reranker and synthesizer. That is LlamaIndex's natural agent shape: an agent that routes between well-built data tools.

LangChain agents are LangGraph graphs. create_agent builds the model-and-tools loop as a graph for you, and middleware hooks into it before and after model calls and tool calls. Middleware is where cross-cutting behaviour goes: summarising long histories, dynamic prompts, model fallbacks, PII redaction, call limits and human approval. When the loop is not enough, you drop to LangGraph and write the state graph yourself, with conditional edges, subgraphs and a supervisor or handoff pattern for multi-agent systems. The agent and the hand-written graph share one runtime, so moving from one to the other does not mean changing frameworks.

Multi-agent support is comparable in reach, different in style. LlamaIndex's AgentWorkflow takes a list of agents, each declaring which others it can_handoff_to, and manages the handoffs; its documentation also shows an orchestrator pattern (sub-agents as tools) and a fully custom planner workflow. LangChain offers subagents-as-tools, LangGraph supervisors and handoffs via Command, and the higher-level Deep Agents package for planning-heavy agents.

How do durability and human approval differ?

This is the section most comparisons skip, and it decides more production projects than retrieval quality does.

LangGraph checkpoints by default once you attach a checkpointer. State is saved after every step to an in-memory, SQLite or Postgres backend. That gives you three things for free: a run can pause for a human and resume hours later, a crashed worker can pick up where it stopped, and you can replay or fork a run from an earlier state when debugging. Human approval in a LangChain agent is one middleware entry:

from langchain.agents import create_agent
from langchain.agents.middleware import HumanInTheLoopMiddleware
from langgraph.checkpoint.memory import InMemorySaver

agent = create_agent(
    model="openai:gpt-4.1",
    tools=[search_policies, issue_refund],
    middleware=[HumanInTheLoopMiddleware(interrupt_on={"issue_refund": True})],
    checkpointer=InMemorySaver(),
)

The issue_refund call now stops and waits for an approve, edit or reject decision, while search_policies runs freely. Swap InMemorySaver for a Postgres checkpointer in production so the pause survives a restart.

LlamaIndex Workflows are ephemeral by default. The durable-workflows guide says so directly: once run() returns, state is gone, and "there is no built-in checkpointer to enable". To survive a restart you serialise the run with Context.to_dict(), store it, and pass Context.from_dict(...) back into run(). To survive a crash mid-run you snapshot as the run progresses, by streaming internal events and saving on each StepStateChanged to NOT_RUNNING. Resume is at-least-once: a step that was executing when the snapshot was taken runs again, so its side effects must be safe to repeat. For stronger guarantees, LlamaIndex documents an integration with DBOS durable execution. Human input uses the built-in InputRequiredEvent and HumanResponseEvent: a tool or step emits the first and waits for the second.

Both approaches work. The difference is who owns the persistence code. With LangGraph it is configuration; with LlamaIndex it is code you write, test and maintain. If your agent issues refunds, deploys infrastructure or sends email, that difference is worth more than any retrieval feature.

How do observability, ecosystem and cost compare?

Observability. LangChain pairs with LangSmith, its commercial tracing and evaluation platform, and also exports OpenTelemetry. LlamaIndex has an instrumentation module with integrations for several tracing tools, including Arize Phoenix and Langfuse. Neither locks you in; the deciding question is which backend your team already runs. Our LangSmith vs Langfuse vs Phoenix comparison covers that choice.

Integrations. Both have hundreds of integrations for models, vector stores and data sources, split into separate packages (langchain-openai, llama-index-llms-openai and so on). LangChain's catalogue is wider for tools and agent infrastructure; LlamaIndex's is deeper for data connectors and readers. Both support Model Context Protocol (MCP) tools.

Cost. The frameworks are free; the model calls are not. The cost difference comes from the pattern you choose, not the library. A 2-step RAG pipeline makes one retrieval and one model call per question. Agentic RAG lets the model decide how many searches to run, which can improve hard answers but makes cost per question variable. Router query engines and supervisor agents each add a model call per routing decision. No head-to-head benchmark was run for this article: measure tokens and latency per question on your own corpus, over several runs, before deciding on cost.

Can you use LlamaIndex and LangChain together?

Yes, and for many teams this is the best answer. The frameworks meet at a clean boundary: a function that takes a question and returns passages.

The usual split is LlamaIndex for the data plane, LangChain for the control plane. LlamaIndex handles parsing, chunking, metadata extraction, indexing and retrieval, because that is where its components are deepest. A LangChain agent, running on LangGraph with checkpoints and approval middleware, calls that retrieval as one of its tools:

from langchain.tools import tool

retriever = index.as_retriever(similarity_top_k=4)  # LlamaIndex index from earlier

@tool
def search_policies(query: str) -> str:
    """Search internal policy documents and return matching passages."""
    nodes = retriever.retrieve(query)
    return "\n\n".join(f"[{n.metadata.get('file_name')}]\n{n.get_content()}" for n in nodes)

Because the boundary is a plain function, you can replace either side later. Keep business logic out of framework-specific classes, pin both frameworks' versions, and test the combined system as one unit, because the two release independently and frequently.

When should you choose LlamaIndex or LangChain?

Choose LlamaIndex when:

• Retrieval quality is the product: knowledge assistants, document Q&A, research over large or messy corpora.
• Your sources include complex PDFs, tables or slide decks and you want control over parsing and chunking.
• You need several retrieval strategies (vector, keyword, graph, routed) behind one interface.
• Your agents are mostly short-lived and route between data tools, so built-in checkpointing matters less.

Choose LangChain when:

• The agent's job is to act: call APIs, write records, run code, with retrieval as one tool among many.
• You need human approval on specific tools, runs that pause for hours, or resume after a crash, without writing persistence code.
• You expect to grow from a single agent into an explicit LangGraph workflow or a multi-agent system on the same runtime.
• You want Python and TypeScript implementations of the same agent model, or already use LangSmith.

Choose both when the application is retrieval-heavy and action-heavy at once, which describes most enterprise assistants. Build the retrieval layer in LlamaIndex and the agent in LangChain, joined by a tool function.

For the wider framework landscape, including Strands, AgentCore and the Microsoft stack, see the AI agent frameworks guide. If you are weighing a framework against a managed service, the LangChain vs Bedrock comparison covers that decision.

How should you evaluate LlamaIndex and LangChain before committing?

Build the same narrow slice of your application in each and measure:

1. Retrieval quality on real questions. Write 30 to 50 questions with known answers from your own documents, and score whether the right passages were retrieved, separately from whether the final answer was right.
2. Ingestion on your worst documents. Run your hardest PDFs and tables through each pipeline and inspect the chunks by hand.
3. Tokens and latency per question, over at least ten runs, including any routing or agent loop calls.
4. Failure behaviour. Kill the process mid-run and check what can be resumed and what repeats.
5. Approval behaviour. Confirm a consequential tool cannot execute without a human decision, and that the pause survives a restart.

The framework that wins on your corpus and your failure tests is the right one, whichever it is.

Frequently Asked Questions

What is the difference between LlamaIndex and LangChain?

LlamaIndex is a data framework built around indexes, retrievers and query engines, with agents implemented as event-driven Workflows. LangChain is an agent framework built around create_agent, a model-and-tools loop with middleware that runs on LangGraph. LlamaIndex goes deeper on RAG; LangChain goes deeper on agent control, persistence and approvals.

Is LlamaIndex better than LangChain for RAG?

For most retrieval-heavy applications, yes. LlamaIndex ships more ready-made RAG components, including ingestion pipelines, node parsers, routers, rerankers, response synthesizers and a property graph index. LangChain supports the same patterns but expects you to assemble more of them yourself. The gap is largest on messy documents and smallest on clean text.

Is LangChain better than LlamaIndex for agent orchestration?

For production agents that need durable state and human approval, generally yes. LangChain agents run on LangGraph, which saves state through a checkpointer after every step and supports interrupts for approval. LlamaIndex Workflows can be made durable, but its documentation states there is no built-in checkpointer, so you write the snapshot and resume logic yourself or use DBOS.

Does LangChain use LangGraph?

Yes. In LangChain 1.x, agents created with create_agent run on the LangGraph runtime, so they share LangGraph's state, checkpointers, streaming and interrupts. You can start with create_agent and move to a hand-written LangGraph graph later without switching frameworks.

Can LlamaIndex and LangChain be used together?

Yes. The common pattern is to build ingestion and retrieval in LlamaIndex and wrap a LlamaIndex retriever or query engine as a tool for a LangChain agent. The boundary is a plain Python function that takes a query and returns passages, so either side can be replaced later.

Are LlamaIndex and LangChain free?

Both open-source frameworks are MIT-licensed and free to use. Each company sells optional services: LlamaIndex offers LlamaParse and hosted document processing, and LangChain offers LangSmith for tracing, evaluation and deployment. Model provider and vector database costs apply either way.

Sources

• LlamaIndex: Building an agent
• LlamaIndex: Multi-agent patterns
• LlamaIndex: Human in the loop
• LlamaIndex Workflows introduction
• LlamaIndex: Writing durable workflows
• LlamaIndex: Property graph index guide
• LlamaIndex GitHub repository
• LangChain agents
• LangChain middleware
• LangChain human-in-the-loop
• LangChain retrieval architectures
• LangGraph persistence

---

Originally published at fp8.co. Subscribe for weekly AI engineering analysis at fp8.co/newsletters.