I Built AgentCost: Real-Time Cost Tracking for LangChain Agents
The Problem
Last month, my LangChain agent cost me $800 in OpenAI fees.
I had no idea which agent was expensive. No idea where to optimize.
I was flying blind.
The Solution
I built AgentCost - an open-source tool that tracks every LLM call your agents make.
How It Works
AgentCost works by intercepting LangChain's LLM calls using Python's monkey patching...
Architecture
Three components:
- Python SDK - Intercepts calls
- FastAPI backend - Stores data
- React dashboard - Visualizes costs
Results
After using AgentCost for 2 weeks:
- Identified that my "Router Agent" was called 10x more than needed
- Switched simple queries to GPT-3.5 instead of GPT-4
- Reduced costs from $800/month to $450/month (44% savings)
Technical Challenges
Monkey patching without breaking user code
How I solved: ...Accurate token counting
The challenge: Different models use different tokenizers...Batching for performance
The solution: Hybrid batching (size + time-based)...
Try It Yourself
AgentCost is open source and free to use:
GitHub: https://github.com/agentcost-ai/agentcost-sdk
Docs: https://agentcost.tech/docs/sdk
pip install agentcost
What's Next
- Cost alerts (Slack/email when threshold hit)
- Automatic optimization suggestions
- OpenAI and Antropic sdk support
Feedback Welcome
If you try AgentCost, I'd love to hear your thoughts!
Twitter: @KushagraA15
GitHub: github.com/agentcost-ai
Top comments (0)