The Evolution of Tool Calling — From Single Function to Multi-Tool Orchestration
Three years ago, "tool calling" meant asking Claude to invoke a single function and parse the JSON response. Today, production agents coordinate dozens of tools across branching execution paths, managing dependencies, handling partial failures, and optimizing for both latency and token budget. This evolution from "call this API" to "orchestrate these 15 tools with typed dependencies" represents one of the most significant architectural shifts in how we build agentic systems. If you're still wiring up tools the way you did in 2024, you're leaving performance on the table and accepting failure modes that modern patterns have solved.
The recent survey "The Evolution of Tool Use in LLM Agents" crystallizes what practitioners have been discovering through trial and error: tool calling has matured through distinct generations, each solving problems the previous generation couldn't address. Understanding these generations isn't academic—it's the difference between agents that work in demos and agents that survive production traffic.
The Three Generations of Tool Calling
The history of tool calling in LLM agents follows a clear progression, each generation emerging from the limitations of its predecessor.
Generation 1 (2023-2024) established the baseline: single-tool invocation with JSON schema descriptions. OpenAI function calling and Anthropic tool use gave us the primitives we still build on today. The pattern was simple—describe your tools in JSON Schema, send the schema with your prompt, parse the function call from the response, execute it, and feed the result back. One tool per turn, synchronous execution, manual result parsing. This worked beautifully for simple use cases: a customer service bot that could check order status, a coding assistant that could run a single command.
The limitation became apparent immediately: no coordination between tools, no dependency awareness. If your agent needed to fetch customer data and then check their order status, that was two round trips minimum. Worse, there was no mechanism for the model to express that these operations had a dependency relationship.
Generation 2 (2024-2025) introduced parallel function calling and basic orchestration. Models could now request multiple tool invocations in a single response, and patterns like the FunctionInvokingChatClient emerged for automatic parallel dispatch. The token cost reduction was immediate—batched requests meant fewer round trips, and fewer round trips meant faster responses and lower costs.
But Generation 2 had its own blind spot: no handling of tool interdependencies or mutable state. When a model requested parallel execution of update_user_profile and send_notification_to_user, nothing in the system understood that these operations might conflict or that one logically should precede the other.
Generation 3 (2026) treats tools as nodes in execution graphs with typed dependencies. This is where modern frameworks have been focusing their energy. Tools declare not just their inputs and outputs but their side effects and dependencies. Execution planners build DAGs from these declarations. Speculative tool execution—running tools before you're certain their results will be needed—becomes possible when you can reason about the graph structure. The ToolGen approach takes this further, representing tools as tokens during training while maintaining their semantic richness at inference time.
The shift between generations isn't merely about capability—it's about where complexity lives. In Generation 1, the complexity was in your application code, manually orchestrating tool calls. In Generation 3, the complexity is in your tool declarations and the framework's execution planner, letting your agent code focus on business logic.
The Tool Selection Problem at Scale
When you have 15 tools, naive approaches work fine. Register them all, let the model pick. But when 15 tools becomes 150—a realistic number for enterprise applications integrating multiple internal APIs—the math breaks down catastrophically.
Consider the token budget: 150 tool descriptions at roughly 200 tokens each consumes 30,000 context tokens before you've even included the user's query. That's most of your context window gone to tool schemas. Research synthesized in the tool use survey confirms what practitioners discovered empirically: accuracy degrades significantly beyond 20-30 registered tools per request. The model simply can't hold that many tool schemas in working memory while also reasoning about which to use.
Hierarchical tool retrieval addresses this through multiple patterns. The AnyTool approach uses self-reflective hierarchical selection for large-scale API pools—essentially, a tool-selection agent that narrows down candidates before the main agent sees them. The Tulip Agent pattern uses embedding-based tool retrieval, computing similarity between the user query and tool descriptions to select a relevant subset before LLM invocation. Re-Invoke rewrites the user's intent into a form optimized for zero-shot tool retrieval.
Dynamic tool registration offers a complementary strategy. Rather than registering all tools upfront, you register only tools relevant to the current conversation state. A "tool routing" agent—often a lightweight, fast model—selects tool subsets before the main agent executes. In practice, this looks like tool categories with lazy loading: register the category descriptions, and only when the agent selects a category do you load the full schemas for tools within it.
The research is clear on one point that practitioners sometimes miss: tool description quality matters more than quantity. The EASYTOOL research on optimizing tool instruction format found that concise, action-oriented descriptions dramatically outperform verbose documentation. "Updates the user's email address in the database" beats a three-paragraph explanation of the email validation rules and database schema. The model needs to understand what the tool does, not how it does it.
Parallel Execution Safety and State Management
Parallel tool execution offers compelling latency improvements, but it introduces failure modes that don't exist in sequential execution. The mutable state problem is the most dangerous: when Tool A and Tool B both modify the same resource, you have race conditions in agent land.
Consider a concrete scenario: the model requests parallel execution of update_user_profile(user_id=123, email="new@example.com") and send_notification_to_user(user_id=123, template="welcome"). In sequential execution, the notification uses the new email address. In parallel execution, it's a race—the notification might go to the old address, or it might fail entirely if the profile update invalidates some intermediate state.
Dependency graph extraction offers a systematic solution. Static analysis of tool schemas can identify read/write conflicts: if Tool A writes to resource X and Tool B reads from resource X, there's an implicit dependency. The LLM Compiler approach treats tool calls as operations in a compiler intermediate representation, with explicit data dependencies that determine execution order. You derive the execution order from the dependency DAG, parallelizing only the tools that are genuinely independent.
Production safeguards layer additional protection:
- Idempotency requirements: Tools eligible for parallel execution must be idempotent—calling them twice with the same inputs produces the same result. This eliminates a class of race conditions entirely.
- Transaction semantics: Checkpoint state before a parallel batch, roll back on any failure. This is easier said than done in distributed systems, but even approximate transactions help.
-
Explicit marking: Tools that modify shared state get an
@exclusivemarker; the executor serializes all exclusive tools while parallelizing the rest.
| Execution Mode | Latency | Failure Handling | Implementation Complexity | Best For |
|---|---|---|---|---|
| Sequential | High (sum of all tool latencies) | Simple—stop on error | Low | Debugging, simple workflows |
| Parallel (naive) | Low (max tool latency) | Complex—partial success states | Medium | Independent, read-only tools |
| Dependency-aware | Medium (critical path latency) | Managed—DAG-aware rollback | High | Production systems with mixed tools |
The dependency-aware approach threads the needle: you get the latency benefits of parallelism where it's safe while maintaining correctness guarantees for tools with real dependencies.
Verification and Error Recovery in Tool Chains
Tool results are not automatically trustworthy. An API might return malformed JSON, a database query might return stale data, an external service might fail silently and return an empty result instead of an error. Verification isn't paranoia—it's engineering rigor.
Chain-of-Abstraction decoding offers an elegant pattern from recent research: the model generates an abstract reasoning chain with placeholders, then populates those placeholders with actual tool outputs. The key insight is that you can verify the reasoning path before invoking tools. If the abstract plan is "Get user's order history [TOOL_RESULT_1], find the most recent order [DERIVED], check its status [TOOL_RESULT_2]," you can validate that this plan makes sense before spending tokens on tool calls.
Tool result validation operates at multiple levels:
- Schema validation: Does the tool output match its declared return type? A tool claiming to return a list of orders should not return a string error message.
- Semantic validation: Does this result make sense given the query? A search for orders in the last week returning orders from 2019 suggests a bug.
- Confidence scoring: Tools can return confidence alongside their results, letting downstream reasoning weight information appropriately.
The research on robustness of agentic function calling identifies common failure modes: tools timing out, returning partial results, or succeeding but with results that don't match the semantic intent. Mitigations include timeout budgets per tool, partial result schemas, and explicit "this tool might return incomplete data" flags in tool descriptions.
Error recovery must go beyond simple retry. Production patterns include:
- Tool-specific fallback chains: If the GitHub API fails, try local git operations. If the primary database is slow, fall back to a read replica.
- Partial result handling: When 3 of 5 parallel tools succeed, the agent should be able to reason with incomplete information rather than failing entirely.
- Escalation policies: Some failures should surface to users (external service down), while others should auto-recover (transient network error). The escalation policy is tool-specific.
Hands-On: Code Walkthrough
Let's build a dependency-aware multi-tool executor in LangGraph. Our scenario: a research agent that needs to fetch company financials, search recent news, analyze sentiment of that news, and generate a summary. The dependency structure is:
- Financials (A) and News Search (B) are independent
- Sentiment Analysis (C) depends on News Search (B)
- Summary (D) depends on both Financials (A) and Sentiment (C)
from typing import Annotated, TypedDict, List, Any
from langchain_core.tools import tool
from langgraph.graph import StateGraph, END
from langgraph.prebuilt import ToolNode
from pydantic import BaseModel, Field
import asyncio
from collections import defaultdict
# Define our tool result types with explicit schemas
class FinancialData(BaseModel):
ticker: str
revenue: float
profit_margin: float
quarter: str
class NewsArticle(BaseModel):
title: str
source: str
published_date: str
snippet: str
class SentimentResult(BaseModel):
overall_sentiment: str # "positive", "negative", "neutral"
confidence: float
article_sentiments: List[dict]
# Tool definitions with explicit dependency hints in docstrings
# The dependency analyzer will parse these annotations
@tool
def fetch_company_financials(ticker: str) -> FinancialData:
"""Fetch quarterly financial data for a company.
Dependencies: none
Side effects: none (read-only)
"""
# Simulated API call - in production, this hits your financial data API
return FinancialData(
ticker=ticker,
revenue=150_000_000,
profit_margin=0.12,
quarter="Q2-2026"
)
@tool
def search_recent_news(company_name: str, days: int = 7) -> List[NewsArticle]:
"""Search for recent news articles about a company.
Dependencies: none
Side effects: none (read-only)
"""
# Simulated search - in production, this calls a news API
return [
NewsArticle(
title=f"{company_name} announces new product line",
source="TechCrunch",
published_date="2026-07-25",
snippet="The company revealed plans for expansion..."
),
NewsArticle(
title=f"{company_name} Q2 earnings beat expectations",
source="Bloomberg",
published_date="2026-07-24",
snippet="Strong performance in cloud division..."
)
]
@tool
def analyze_news_sentiment(articles: List[dict]) -> SentimentResult:
"""Analyze sentiment of news articles.
Dependencies: requires news articles (output of search_recent_news)
Side effects: none (read-only)
"""
# In production, this runs sentiment analysis model
return SentimentResult(
overall_sentiment="positive",
confidence=0.85,
article_sentiments=[
{"title": a.get("title", ""), "sentiment": "positive"}
for a in articles
]
)
@tool
def generate_research_summary(
financials: dict,
sentiment: dict,
company_name: str
) -> str:
"""Generate a research summary combining financial data and news sentiment.
Dependencies: requires financials and sentiment analysis results
Side effects: none (read-only)
"""
return f"""
Research Summary: {company_name}
Financial Highlights:
- Revenue: ${financials.get('revenue', 0):,.0f}
- Profit Margin: {financials.get('profit_margin', 0):.1%}
- Quarter: {financials.get('quarter', 'N/A')}
News Sentiment: {sentiment.get('overall_sentiment', 'unknown').upper()}
(Confidence: {sentiment.get('confidence', 0):.0%})
Overall Assessment: Based on strong financials and positive news sentiment,
the outlook appears favorable.
"""
# Dependency analyzer that builds execution DAG from tool schemas
class ToolDependencyAnalyzer:
"""Analyzes tool dependencies and builds execution graph."""
def __init__(self, tools: List):
self.tools = {t.name: t for t in tools}
self.dependencies = self._parse_dependencies()
def _parse_dependencies(self) -> dict[str, List[str]]:
"""Extract dependencies from tool docstrings.
In production, you'd use more sophisticated parsing or explicit
decorators. This demonstrates the pattern.
"""
deps = defaultdict(list)
# Hardcoded for clarity - in production, parse from docstrings/annotations
deps["analyze_news_sentiment"] = ["search_recent_news"]
deps["generate_research_summary"] = [
"fetch_company_financials",
"analyze_news_sentiment"
]
# Tools with no dependencies are implicitly independent
return dict(deps)
def get_execution_order(self, requested_tools: List[str]) -> List[List[str]]:
"""Return tools grouped by execution level.
Tools in the same group can execute in parallel.
Groups must execute sequentially.
"""
remaining = set(requested_tools)
completed = set()
execution_levels = []
while remaining:
# Find tools whose dependencies are all completed
ready = {
tool for tool in remaining
if all(dep in completed for dep in self.dependencies.get(tool, []))
}
if not ready:
raise ValueError(f"Circular dependency detected in {remaining}")
execution_levels.append(list(ready))
completed.update(ready)
remaining -= ready
return execution_levels
# State definition for our research agent
class ResearchState(TypedDict):
company_name: str
ticker: str
financials: dict | None
news_articles: List[dict] | None
sentiment: dict | None
summary: str | None
current_level: int
execution_plan: List[List[str]]
errors: List[str]
# Parallel tool executor with dependency awareness
class DependencyAwareToolExecutor:
"""Executes tools respecting dependency order with parallel dispatch."""
def __init__(self, tools: List):
self.tools = {t.name: t for t in tools}
self.analyzer = ToolDependencyAnalyzer(tools)
async def execute_level(
self,
tool_names: List[str],
state: ResearchState
) -> dict[str, Any]:
"""Execute a single level of tools in parallel."""
async def run_tool(name: str) -> tuple[str, Any, str | None]:
tool = self.tools[name]
try:
# Map tool names to their required arguments from state
args = self._get_tool_args(name, state)
result = await asyncio.to_thread(tool.invoke, args)
return (name, result, None)
except Exception as e:
return (name, None, str(e))
# Execute all tools in this level concurrently
tasks = [run_tool(name) for name in tool_names]
results = await asyncio.gather(*tasks)
return {
name: {"result": result, "error": error}
for name, result, error in results
}
def _get_tool_args(self, tool_name: str, state: ResearchState) -> dict:
"""Map state to tool arguments. Production code would be more dynamic."""
if tool_name == "fetch_company_financials":
return {"ticker": state["ticker"]}
elif tool_name == "search_recent_news":
return {"company_name": state["company_name"], "days": 7}
elif tool_name == "analyze_news_sentiment":
return {"articles": state["news_articles"]}
elif tool_name == "generate_research_summary":
return {
"financials": state["financials"],
"sentiment": state["sentiment"],
"company_name": state["company_name"]
}
return {}
# Build the LangGraph workflow
def build_research_graph():
"""Construct the LangGraph StateGraph for research workflow."""
tools = [
fetch_company_financials,
search_recent_news,
analyze_news_sentiment,
generate_research_summary
]
executor = DependencyAwareToolExecutor(tools)
analyzer = ToolDependencyAnalyzer(tools)
async def plan_execution(state: ResearchState) -> ResearchState:
"""Generate execution plan based on dependencies."""
all_tools = [
"fetch_company_financials",
"search_recent_news",
"analyze_news_sentiment",
"generate_research_summary"
]
plan = analyzer.get_execution_order(all_tools)
return {**state, "execution_plan": plan, "current_level": 0}
async def execute_current_level(state: ResearchState) -> ResearchState:
"""Execute tools at current dependency level."""
level = state["current_level"]
tools_to_run = state["execution_plan"][level]
results = await executor.execute_level(tools_to_run, state)
# Update state with results
new_state = {**state}
errors = list(state.get("errors", []))
for tool_name, outcome in results.items():
if outcome["error"]:
errors.append(f"{tool_name}: {outcome['error']}")
else:
# Map results to state fields
if tool_name == "fetch_company_financials":
new_state["financials"] = outcome["result"].dict()
elif tool_name == "search_recent_news":
new_state["news_articles"] = [a.dict() for a in outcome["result"]]
elif tool_name == "analyze_news_sentiment":
new_state["sentiment"] = outcome["result"].dict()
elif tool_name == "generate_research_summary":
new_state["summary"] = outcome["result"]
new_state["current_level"] = level + 1
new_state["errors"] = errors
return new_state
def should_continue(state: ResearchState) -> str:
"""Check if more execution levels remain."""
if state["current_level"] >= len(state["execution_plan"]):
return "done"
return "continue"
# Build the graph
graph = StateGraph(ResearchState)
graph.add_node("plan", plan_execution)
graph.add_node("execute", execute_current_level)
graph.set_entry_point("plan")
graph.add_edge("plan", "execute")
graph.add_conditional_edges(
"execute",
should_continue,
{"continue": "execute", "done": END}
)
return graph.compile()
# Example usage
async def run_research(company: str, ticker: str):
graph = build_research_graph()
initial_state: ResearchState = {
"company_name": company,
"ticker": ticker,
"financials": None,
"news_articles": None,
"sentiment": None,
"summary": None,
"current_level": 0,
"execution_plan": [],
"errors": []
}
result = await graph.ainvoke(initial_state)
return result
# Run it
if __name__ == "__main__":
result = asyncio.run(run_research("Acme Corp", "ACME"))
print(result["summary"])
print(f"Errors: {result['errors']}")
This implementation demonstrates several key patterns:
- Explicit dependency declaration in tool docstrings (production systems would use decorators or schema annotations)
- Topological sorting to determine execution levels
-
Parallel dispatch within levels using
asyncio.gather - Partial failure handling that continues execution when some tools fail
- State mapping between tool outputs and agent state
The execution order for our research scenario is:
- Level 0:
fetch_company_financials,search_recent_news(parallel) - Level 1:
analyze_news_sentiment(waits for news) - Level 2:
generate_research_summary(waits for financials and sentiment)
What This Means for Your Stack
The evolution from single-tool calling to multi-tool orchestration has concrete implications for how you architect agentic systems.
Token budget planning becomes an architectural concern. Just as you budget context window space for RAG retrieval results, you must now budget for tool descriptions. A production system might allocate: 4K tokens for system prompt, 8K for conversation history, 6K for RAG context, and 4K for tool schemas—leaving 10K for actual reasoning. With large toolsets, consider tool description compression, summarization, or the hierarchical retrieval patterns discussed earlier. Monitor token costs per tool invocation in LangSmith traces to identify tools with bloated descriptions.
Tool API design must optimize for agent consumption. This is distinct from designing APIs for human developers:
- Idempotent operations wherever possible: Agents retry. A lot. Tools that can be safely re-invoked simplify error handling dramatically.
- Clear input/output schemas: Pydantic models, not loose dicts. The model needs to understand what it's getting back.
- Explicit side effect documentation: If a tool sends an email, that must be crystal clear in the description. Agents can't infer side effects.
Framework selection depends on your orchestration needs. The landscape has matured considerably:
| Framework | Tool Orchestration Model | Best For |
|---|---|---|
| LangGraph | Full control via custom nodes | Complex, custom workflows |
| Microsoft AutoGen | Built-in parallel calling, scale-focused architecture | Multi-agent systems |
| CrewAI | Task-level parallelism | Role-based agent teams |
Migration checklist for existing tool implementations:
- Audit every tool for idempotency and side effects. Document findings.
- Add explicit type annotations (Pydantic models) to all tool signatures.
- Implement a result validation layer between tools and agent reasoning.
- Classify tools as independent vs. stateful for parallel execution eligibility.
- Set up tool-specific error handlers before enabling parallel execution.
- Add dependency declarations to tools that consume other tools' outputs.
The research on agentic system architecture suggests these patterns will become table stakes. Systems built with sequential-only tool calling will hit performance ceilings that dependency-aware orchestration avoids entirely.
What to Build This Week
Build a "tool catalog" service for your existing agent. Take your current tool implementations and build a metadata layer that includes:
- Dependency declarations (what other tools' outputs does this tool need?)
- Side effect classifications (read-only, creates resource, updates resource, deletes resource)
- Idempotency flags (safe to retry? safe to parallelize?)
- Estimated latency and token cost per invocation
Then implement a simple execution planner that reads this metadata and generates dependency-aware execution orders. Start with just visualization—show yourself the DAG your current tool calls would produce. You'll likely discover implicit dependencies you hadn't thought about and opportunities for parallelization you've been missing.
The goal isn't to replace your current execution model immediately. The goal is to understand your tool graph well enough to make informed decisions about when parallel execution is safe and when it isn't. That understanding will inform every tool you write going forward.
Sources
- The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestration
- AI tool calling - .NET | Microsoft Learn
- LangChain and LangGraph Agent Frameworks Reach v1.0
- LangGraph overview - Docs by LangChain
- The Agent Development Lifecycle: Build, Test, Deploy
- The best AI agent frameworks in 2026
- GitHub - microsoft/autogen: A programming framework for agentic AI
- AutoGen v0.4: Reimagining the foundation of agentic AI for scale and more - Microsoft Research
- crewAIInc/crewAI: Framework for orchestrating role-playing agents
- ai-agent-papers/capability-papers/tool-use.md - GitHub
- GitHub - Applied-Machine-Learning-Lab/Awesome-Function-Callings
- The Evolution of Agentic AI Software Architecture
This is part of the **Agentic Engineering Weekly* series — a deep-dive every Monday into the frameworks,
patterns, and techniques shaping the next generation of AI systems.*
Follow the Agentic Engineering Weekly series on Dev.to to catch every edition.
Building something agentic? Drop a comment — I'd love to feature reader projects.
Top comments (0)