DEV Community

Alex Spinov
Alex Spinov

Posted on

I Tested 5 AI Agent Frameworks — Here's Which One Actually Works

#webdev #ai #python #discuss

AI agents are the hottest thing in tech right now. Every framework promises autonomous AI that can browse the web, write code, and make decisions.

I tested 5 of them on the same task: research a topic, find relevant APIs, and write a summary with working code examples. Here's what actually worked.

The Contenders

  1. AutoGen (Microsoft) — multi-agent conversations
  2. CrewAI — role-based agent teams
  3. LangGraph — stateful agent graphs
  4. Phidata — production AI assistants
  5. Raw API calls — just Claude/GPT with function calling

Results

Framework Task Completed? Time Code Quality Setup Time
AutoGen Partial 4 min Medium 30 min
CrewAI Yes 3 min Good 15 min
LangGraph Yes 5 min Good 45 min
Phidata Yes 2 min Good 10 min
Raw API Yes 1 min Best 5 min

What I Learned

1. Raw API calls beat every framework for simple tasks

If your agent needs to do one thing well (search, summarize, extract), just use the API directly:

import anthropic

client = anthropic.Anthropic()

response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    tools=[{
        "name": "search_api",
        "description": "Search for free APIs",
        "input_schema": {
            "type": "object",
            "properties": {
                "query": {"type": "string"}
            },
            "required": ["query"]
        }
    }],
    messages=[{"role": "user", "content": "Find free APIs for weather data"}]
)
Enter fullscreen mode Exit fullscreen mode

No framework overhead. No abstractions to learn. Just function calling.

2. CrewAI has the best developer experience

If you DO need multiple agents working together, CrewAI's role-based approach is intuitive:

from crewai import Agent, Task, Crew

researcher = Agent(
    role="API Researcher",
    goal="Find free APIs for the given topic",
    backstory="Expert at finding developer tools"
)

writer = Agent(
    role="Technical Writer",
    goal="Write code examples using the APIs found",
    backstory="Writes clear, practical tutorials"
)
Enter fullscreen mode Exit fullscreen mode

3. LangGraph is powerful but over-engineered for most use cases

LangGraph gives you full control over agent state and transitions. But for 90% of tasks, you don't need state machines — you need a prompt and a tool.

4. MCP is the future of agent tooling

The Model Context Protocol (MCP) lets agents connect to tools through a standard interface. Instead of writing custom integrations, you connect MCP servers:

  • File system access
  • Database queries
  • API calls
  • Browser automation

I've curated a collection of 15+ MCP servers that work out of the box.

My Recommendation 

Single agent, simple task → Raw API with function calling
Multiple agents, clear roles → CrewAI
Complex state management → LangGraph
Quick prototype → Phidata
Enter fullscreen mode Exit fullscreen mode

For a comprehensive list of agent tools, I put together Awesome AI Agents — frameworks, MCP tools, code agents, platforms, and evaluation tools.

What agent framework are you using? Has anyone actually deployed agents in production (not just demos)?

More resources:

Top comments (0)