Building an AI Agent is like cooking. You can have the best ingredients (LLMs), but if your kitchen (tech stack) is a mess, the final dish will be bad.
I've been iterating on AI Agents for months. Today, I'm sharing my "ultimate stack"—the tools I use for development, testing, deployment, and monitoring.
1. The Brain: LLM APIs
- For Complex Reasoning: OpenAI's
gpt-4oor Anthropic'sclaude-3.5-sonnet. (Don't use 3.5-turbo for agents, it fails on tool calling). - For Fast & Cheap Tasks:
Llama 3 8Bvia Groq. The token generation speed is insane and perfect for simple routing tasks.
2. The Framework: LangGraph
As I mentioned in a previous article, I prefer LangGraph over standard LangChain. The state-machine model gives me the deterministic control I need so my agents don't spiral into infinite loops.
3. The Memory: Supabase + pgvector
Instead of setting up a whole new vector database, I use Supabase. It provides a Postgres database with vector extensions built-in. It handles both my user authentication and my RAG memory.
4. The Observability: Langfuse
You cannot debug an Agent using print() statements. Langfuse is an open-source tool that traces every LLM call, tool call, and prompt. It shows you exactly where the Agent failed and how many tokens you burned.
5. The API Router: OpenRouter / LiteLLM
If you are like me and constantly switch between GPT, Claude, and open-source models, don't write new code for each API. Use OpenRouter. One API key, hundreds of models. (Bonus: It handles rate limits gracefully).
6. The Evaluation: Ragas
How do you know if your Agent is getting better or worse? Ragas is an evaluation framework that tests your RAG pipelines and agents on metrics like "faithfulness" and "answer relevance".
The Stack Summarized:
User Input -> Next.js -> LangGraph (Orchestrator) -> Groq/OpenAI -> Supabase (Memory) -> Langfuse (Monitoring)
What does your stack look like? Am I missing any game-changing tools? Let me know below!
Top comments (0)