I've been building agents for a while and one thing keeps bugging me: for a tool-using agent (ReAct / LangGraph / OpenAI tools), the expensive part isn't the final answer — it's the planning loop. All those back-and-forth LLM calls to figure out which tools to call in what order. And we throw that plan away after every task.
So the 50th time the agent does "pull active users → filter → export CSV", it re-derives the whole plan from scratch and pays full LLM cost again. Same workflow, slightly different inputs, full price, every time. And it's non-deterministic, so sometimes it takes a dumb path.
Existing memory tools (Mem0, Zep, vector RAG) mostly store text/facts — "what the user said." They don't capture the procedure that worked and replay it.
A few honest questions for people running agents in prod:
- What actually happens the second/50th time your agent hits the same kind of task? Does it re-plan?
- Roughly what fraction of your agent traffic is repetitive / same-shape-different-values?
- Have you already hacked around this (growing "lessons" prompt, few-shot skill file, caching)? What broke?
- If something cut your planning/LLM calls ~30–45% at the same success rate — would that be worth paying?
Not selling anything, genuinely trying to validate whether this is a real, expensive, top-of-mind problem.
Top comments (1)
Repeated re-planning is one of those small wastes that becomes a product smell. If the task pattern is stable, the system should remember the workflow shape, not the stale facts. Plans can be reusable while inputs and constraints stay fresh.