I've been running a multi-agent LangGraph pipeline for a few weeks now and kept hitting the same two walls: redundant API calls inflating costs, and agent crashes the moment any upstream service became flaky.
My setup had 4 agents calling overlapping tools. Under real load, I was making 40+ calls per run to endpoints that returned the same data within a 30-minute window. My circuit-breaker logic was copy-pasted and inconsistent across tools. Observability was basically nonexistent — when something broke, I had no structured signal to debug from.
I found ToolOps and the core idea is sound: treat every tool call the way infrastructure engineers treat microservice communication. The @readonly / @sideeffect decorator split is opinionated in a good way — it forces you to be explicit about whether a tool call is idempotent or not, which turns out to matter a lot when you're deciding what's safe to cache and retry.
What stuck with me practically:
- Request coalescing: in a 50-concurrent-call benchmark from their docs, 50 calls collapsed to 1 upstream request. I tested something smaller but directionally got the same behavior — the thundering herd problem on cache miss is real and this handles it cleanly.
- Semantic caching: I was skeptical but the intent-matching approach is genuinely useful for NLP tool inputs. "Check status of invoice #442" and "Is invoice 442 paid?" hitting the same cache entry reduced LLM token usage noticeably.
- toolops doctor: a single CLI command that validates all your backends and reports circuit breaker state. Small thing, but it's exactly what you want to wire into a health check endpoint.
The integration is one decorator per function. Zero changes to business logic. Works across LangGraph, CrewAI, LlamaIndex, and MCP natively.
It's early — the web dashboard and budget control features are still on the roadmap — but the core resilience layer is solid. Apache 2.0, built by Hedi Manai (@hedimanai on LinkedIn).
GitHub: https://github.com/hedimanai-pro/toolops
Docs: https://hedimanai.vercel.app/projects/toolops.html
Top comments (2)
@bessiegannon Hey Bessie, thank you so much for taking the time to test ToolOps and write this detailed post!
I'm really glad it helped solve your thundering herd problem and reduced those 40+ redundant calls. Your feedback on request coalescing, semantic caching, and the
@readonly/@sideeffectdecorators is super valuable.Really appreciate you sharing your real-world experience with the community 🙏
Hey! Thank you so much for your comment, it really made my day ❤️
I was honestly stuck with those 40+ repeated calls, and ToolOps turned out to be such an elegant solution. Being able to clearly mark
@readonly/@sideeffectmakes a huge difference.I’m definitely going to keep using it on my other agents. Really great tool, and the documentation is excellent too! Thank you for building this