DEV Community

Cover image for Comparing AI Agents Python Library Options for Production
Ayush Kumar
Ayush Kumar

Posted on • Originally published at logiclooptech.dev

Comparing AI Agents Python Library Options for Production

#ai #agents #python #llm

Quick answer

If you need a Python library to build an AI agent that can run in production, start with LangChain for flexibility, CrewAI for team-style orchestration, or LlamaIndex if your focus is on data-centric retrieval. All three install with a single pip command, but they differ in abstraction level, runtime cost, and how easy they are to test inside a FastAPI service.

What are the top Python libraries for building AI agents?

The three most battle-tested options today are LangChain, CrewAI, and LlamaIndex.

  • LangChain gives you granular control over prompts, memory, and tool integration. It’s a good fit when you want to stitch together many LLM calls or custom utilities.
  • CrewAI treats an agent as a small “crew” of specialists. It’s built around task decomposition, role assignment, and result aggregation. Think of it as a higher-level wrapper that reduces boilerplate for multi-step workflows.
  • LlamaIndex (formerly GPT Index) shines when your agent needs to search or summarize large document collections. It builds indexes on top of vector stores and lets you query them with natural language.

All three are open source, support OpenAI, Anthropic, and Azure endpoints, and have async APIs that play nicely with FastAPI.

How do I install and set up LangChain vs CrewAI vs LlamaIndex?

Installation is straightforward, but each library pulls a different set of optional dependencies.

# Core libraries
pip install langchain crewai llama-index
# Optional LLM providers (pick what you need)
pip install openai anthropic
# Vector store for LlamaIndex example
pip install chromadb
Enter fullscreen mode Exit fullscreen mode

LangChain minimal setup 

from langchain.llms import OpenAI
from langchain.prompts import PromptTemplate
from langchain.chains import LLMChain

llm = OpenAI(model="gpt-4o-mini")
prompt = PromptTemplate.from_template("Translate this to French: {text}")
chain = LLMChain(llm=llm, prompt=prompt)
Enter fullscreen mode Exit fullscreen mode

CrewAI minimal setup 

from crewai import Agent, Task, Crew

# Define a simple research agent
researcher = Agent(
    role="Researcher",
    goal="Find the latest Python web frameworks",
    backstory="You are a senior backend engineer.",
    llm="gpt-4o-mini",
)

task = Task(
    description="List three frameworks with a one‑sentence summary each.",
    agent=researcher,
)

crew = Crew(agents=[researcher], tasks=[task])
Enter fullscreen mode Exit fullscreen mode

LlamaIndex minimal setup 

from llama_index import VectorStoreIndex, SimpleDirectoryReader
from llama_index.llms import OpenAI

documents = SimpleDirectoryReader("data/").load_data()
index = VectorStoreIndex.from_documents(documents, llm=OpenAI(model="gpt-4o-mini"))
Enter fullscreen mode Exit fullscreen mode

All three snippets run in a fresh virtual environment. The biggest gotcha I’ve hit is version mismatches between langchain and crewai when they both depend on openai. Pinning openai>=1.0,<2.0 in requirements.txt saved me a day of debugging.

Can you show a step-by-step example of a simple AI agent with each library?

Below is a tiny “weather lookup” agent that takes a city name, calls a mock weather API, and returns a friendly sentence. The logic is the same; only the orchestration differs.

LangChain version 

import httpx
from langchain.llms import OpenAI
from langchain.prompts import PromptTemplate
from langchain.chains import LLMChain

llm = OpenAI(model="gpt-4o-mini")
prompt = PromptTemplate.from_template(
    "You are a helpful assistant. Use the JSON response {weather} to answer: "
    "What is the weather like in {city}?"
)

async def fetch_weather(city: str) -> dict:
    async with httpx.AsyncClient() as client:
        resp = await client.get(f"https://api.example.com/weather?city={city}")
        return resp.json()

async def weather_agent(city: str) -> str:
    weather = await fetch_weather(city)
    chain = LLMChain(llm=llm, prompt=prompt)
    return await chain.arun(weather=weather, city=city)
Enter fullscreen mode Exit fullscreen mode

CrewAI version 

from crewai import Agent, Task, Crew
import httpx

weather_agent = Agent(
    role="Weather Bot",
    goal="Provide a short weather summary",
    backstory="You are a backend engineer who loves concise messages.",
    llm="gpt-4o-mini",
)

async def fetch_weather(city):
    async with httpx.AsyncClient() as client:
        r = await client.get(f"https://api.example.com/weather?city={city}")
        return r.json()

task = Task(
    description="Summarize the JSON weather data for {city}",
    expected_output="A one‑sentence weather report.",
    agent=weather_agent,
    async_func=fetch_weather,   # CrewAI lets you attach a callable
)

crew = Crew(agents=[weather_agent], tasks=[task])

async def run_crew(city):
    return await crew.kickoff(inputs={"city": city})
Enter fullscreen mode Exit fullscreen mode

LlamaIndex version 

from llama_index import SimpleDirectoryReader, VectorStoreIndex, ServiceContext
from llama_index.llms import OpenAI
import httpx

# Build a trivial index containing a prompt template
documents = SimpleDirectoryReader(input_files=["prompt.txt"]).load_data()
service_context = ServiceContext.from_defaults(llm=OpenAI(model="gpt-4o-mini"))
index = VectorStoreIndex.from_documents(documents, service_context=service_context)

async def fetch_weather(city):
    async with httpx.AsyncClient() as client:
        r = await client.get(f"https://api.example.com/weather?city={city}")
        return r.json()

async def llama_agent(city):
    query = f"Summarize this JSON: {await fetch_weather(city)}"
    response = await index.as_query_engine().aquery(query)
    return response.response
Enter fullscreen mode Exit fullscreen mode

All three agents return a string like “In Paris it’s 12 °C with light rain.” The LangChain version feels most explicit; CrewAI hides the prompt plumbing but requires you to think in terms of tasks; LlamaIndex bundles the prompt inside an index, which can be overkill for a single API call.

How do the libraries compare on performance and features?

Feature LangChain CrewAI LlamaIndex
Prompt control Full, low-level Medium (templates per task) Low (index-driven)
Tool integration Easy via Tool class Built-in task-to-tool mapping Requires custom retriever
Memory Built-in ConversationBufferMemory Implicit via task history Index acts as persistent memory
Async support Native async methods Async tasks supported Async query engine
Cost predictability You manage each LLM call Crew may fire extra calls for coordination Index creation can add one-off token cost
Learning curve Steeper, many concepts Gentle, opinionated Gentle if you only need retrieval

In my production services, LangChain adds ~15 ms per extra tool call, CrewAI adds ~30 ms overhead for task orchestration, and LlamaIndex adds a one-time indexing cost that can be several seconds for a 10 k document corpus. If you’re budget-conscious, watch out for CrewAI’s hidden “role-play” calls – they can double your token bill if you forget to set max_tokens on the underlying LLM.

How do I integrate the agent into a FastAPI service?

FastAPI is already async-first, so you can drop any of the async agents directly into a route. Below is a minimal FastAPI app that uses the LangChain weather agent; swap the import for crew_agent or llama_agent to try the other libraries.

# app/main.py
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import uvicorn

# Choose your agent implementation
from agents.langchain_weather import weather_agent  # or crew_agent, llama_agent

app = FastAPI(title="Weather AI Agent")

class CityRequest(BaseModel):
    city: str

@app.post("/weather")
async def get_weather(req: CityRequest):
    try:
        result = await weather_agent(req.city)
        return {"city": req.city, "report": result}
    except Exception as exc:
        raise HTTPException(status_code=502, detail=str(exc))

if __name__ == "__main__":
    uvicorn.run("app.main:app", host="0.0.0.0", port=8000, reload=True)
Enter fullscreen mode Exit fullscreen mode

Production tips

  • Timeouts – LLM calls can stall. Configure httpx timeouts (client = httpx.AsyncClient(timeout=10.0)).
  • Rate limits – Put a simple token bucket middleware around the route; otherwise you’ll hit OpenAI’s per-minute caps.
  • Observability – Log the raw LLM request/response (scrub PII) and emit Prometheus metrics for latency and token usage.
  • Cold start – Load the LLM client once at startup, not per request. I once saw a 2-second spike when I instantiated OpenAI() inside the route handler.

If you need more context, see my earlier post on AI agent python ollama: Build, Test, Deploy with FastAPI for a full deployment pipeline.

What testing and validation strategies work for production agents?

Testing LLM-driven code feels weird because the output is nondeterministic. Here’s what I rely on:

  1. Unit-level prompt snapshots – Store the exact prompt string and expected token count. Use assert "Translate this to French" in prompt. If the prompt changes, the test fails before you even call the model.
  2. Mocked LLM client – Replace OpenAI with a stub that returns a deterministic JSON payload. Libraries like pytest-mock let you patch OpenAI.__call__. This isolates the business logic from the provider.
  3. Contract tests on the FastAPI endpoint – Use httpx.AsyncClient(app=app) in pytest to hit /weather with a fake city and assert the shape of the JSON response.
  4. Canary deployment – Roll out the new agent to 5 % of traffic, compare latency and error rates against the stable version. If the canary’s error rate exceeds 0.5 %, roll back automatically.
  5. Cost guardrails – Write a small background job that queries the OpenAI usage API every hour and alerts when the daily quota is near the limit. I once had a runaway loop that generated 2 M tokens in an hour because a missing max_tokens parameter let the model keep streaming.

When you’re ready to scale, read the “Testing and validation” chapter of the official LangChain docs – they outline a “LLMMock” helper that integrates nicely with pytest.

FAQ

Which library should I start with for a simple chatbot?

LangChain gives the most flexibility and the smallest dependency footprint for a single-turn bot. If you want built-in task management, try CrewAI.

Do I need a vector store for every agent?

Only if you plan to retrieve or embed large text collections. LlamaIndex shines there; otherwise it adds unnecessary latency.

Can I swap OpenAI for an on-prem model?

All three libraries expose a generic LLM interface. Point the client to your local server (e.g., OpenAI(api_base="http://localhost:8000/v1")) and the rest of the code stays the same.

How do I monitor token usage per request?

Wrap the LLM call in a decorator that reads the usage field from the response and pushes it to Prometheus or CloudWatch. I keep a request_id in the FastAPI state to correlate logs.

Key Takeaways

  • LangChain, CrewAI, and LlamaIndex are the three most production-ready ai agents python library options.
  • Choose LangChain for fine-grained control, CrewAI for multi-step orchestration, and LlamaIndex for retrieval-heavy workloads.
  • Installation is a single pip install line; watch version pins for openai.
  • Each library can be wrapped in an async FastAPI endpoint with minimal boilerplate.
  • Test prompts, mock LLM calls, and use canary releases to keep production stable.
  • Keep an eye on latency, token cost, and rate limits – they’re the hidden failure points that bite most engineers.

Top comments (0)