My LangGraph agent was hammering the same API endpoints 40 per run. Solved it with ToolOps

Bessie Gannon — Tue, 12 May 2026 20:23:47 +0000

I've been running a multi-agent LangGraph pipeline for a few weeks now and kept hitting the same two walls: redundant API calls inflating costs, and agent crashes the moment any upstream service became flaky.

My setup had 4 agents calling overlapping tools. Under real load, I was making 40+ calls per run to endpoints that returned the same data within a 30-minute window. My circuit-breaker logic was copy-pasted and inconsistent across tools. Observability was basically nonexistent — when something broke, I had no structured signal to debug from.

I found ToolOps and the core idea is sound: treat every tool call the way infrastructure engineers treat microservice communication. The @readonly / @sideeffect decorator split is opinionated in a good way — it forces you to be explicit about whether a tool call is idempotent or not, which turns out to matter a lot when you're deciding what's safe to cache and retry.

What stuck with me practically:

Request coalescing: in a 50-concurrent-call benchmark from their docs, 50 calls collapsed to 1 upstream request. I tested something smaller but directionally got the same behavior — the thundering herd problem on cache miss is real and this handles it cleanly.
Semantic caching: I was skeptical but the intent-matching approach is genuinely useful for NLP tool inputs. "Check status of invoice #442" and "Is invoice 442 paid?" hitting the same cache entry reduced LLM token usage noticeably.
toolops doctor: a single CLI command that validates all your backends and reports circuit breaker state. Small thing, but it's exactly what you want to wire into a health check endpoint.

The integration is one decorator per function. Zero changes to business logic. Works across LangGraph, CrewAI, LlamaIndex, and MCP natively.

It's early — the web dashboard and budget control features are still on the roadmap — but the core resilience layer is solid. Apache 2.0, built by Hedi Manai (@hedimanai on LinkedIn).

GitHub: https://github.com/hedimanai-pro/toolops
Docs: https://hedimanai.vercel.app/projects/toolops.html

DEV Community: Bessie Gannon

My LangGraph agent was hammering the same API endpoints 40 per run. Solved it with ToolOps