Why Agent Evaluation Is Different
Evaluating a RAG system, the final answer is the evaluation target — correct or incorrect, accurate or not.
An Agent's process is also under evaluation. Taking 3 extra steps to reach a correct answer, and calling the right tool with wrong arguments — both are failures that only final-answer evaluation misses.
Agent evaluation needs two dimensions:
Dimension 1: Tool Call Accuracy (automatable)
Tool name accuracy: did it call the right tool?
Parameter accuracy: were the arguments correct?
Step efficiency: how many steps did it take vs the minimum needed?
Dimension 2: Trajectory Quality (LLM-as-Judge)
Reasoning soundness: was each step logically justified?
No redundant calls: did it avoid unnecessary tool invocations?
Error handling: when a tool returned nothing, did it adapt?
Experiment Design
Agent under test: customer support assistant with 3 tools:
search_faq(query) # search product FAQ
get_order_status(order_id) # fetch order status
calculate_refund(amount, days_since_purchase) # compute refund amount
15 test cases across three categories:
| Category | Count | Example |
|---|---|---|
| Simple (1 tool) | 9 | "What's the refund policy?" → search_faq
|
| Multi-step (2 tools) | 3 | "How much refund for ORD-004?" → get_order_status + calculate_refund
|
| Edge cases | 3 | Non-existent order, cancelled order, no-tool greeting |
Each case has ground truth: expected tool call sequence and arguments.
Results
Tool Call Accuracy
[T01] ✓ tools=['search_faq'] name=✓ params=100%
[T02] ✓ tools=['get_order_status'] name=✓ params=100%
[T03] ✗ tools=[] expected=['search_faq'] name=✗ params=100%
[T04] ✗ tools=[] expected=['get_order_status',...] name=✗ params=100%
[T05] ✓ tools=['calculate_refund'] name=✓ params=100%
[T06] ✓ tools=['get_order_status','calculate_refund'] name=✓ params=100%
...
[T10] ✗ tools=[] expected=['search_faq'] name=✗ params=100%
[T13] ✓ tools=[] expected=[] (greeting, no tool) name=✓ params=100%
[T14] ✗ tools=[] expected=['get_order_status'] name=✗ params=100%
Summary Metrics
Tool Name Accuracy: 73% (11/15 correct sequences)
Parameter Accuracy: 100% (avg across all tool calls)
Step Efficiency: 0.73x (1.0 = optimal)
Trajectory Quality: 3.73/5 (LLM-as-Judge)
Failed cases (4):
[T03] expected=['search_faq'] actual=[]
[T04] expected=['get_order_status', 'calculate_refund'] actual=[]
[T10] expected=['search_faq'] actual=[]
[T14] expected=['get_order_status'] actual=[]
Per-category breakdown:
Simple (1 tool) tool_acc=75% trajectory=3.75/5
Multi-step (2 tools) tool_acc=67% trajectory=3.67/5
Edge cases tool_acc=67% trajectory=3.67/5
Three Findings
Finding 1: All Failures Are "Should Have Called But Didn't"
All 4 failures share the same pattern: the Agent answered directly without calling any tool. Parameter accuracy at 100% confirms that whenever the Agent decided to call a tool, the arguments were correct.
Failure type distribution:
Wrong tool selected (called but wrong choice): 0
Wrong parameters (right tool, wrong args): 0
Tool not triggered (should call but didn't): 4
Tool execution logic has no problem. Tool triggering logic is the bottleneck. In those four cases the Agent concluded it already knew the answer and skipped the call.
T03 ("How long does shipping take?") and T10 ("What are member benefits?"): glm-4-flash has built-in knowledge on these topics and answered directly even with the instruction to use tools. T04 (compound question needing two steps) and T14 (cancelled order refund): the Agent inferred rather than queried.
Fix direction: add stronger per-scenario constraints to the System Prompt, or use a rule-based routing layer to handle specific question types before the LLM makes a decision.
Finding 2: Step Efficiency 0.73x — Below 1.0 Because Steps Were Skipped
Step efficiency below 1.0 might seem like a good thing ("fewer steps"), but the actual cause here is that multi-step tasks had zero tool calls instead of the required number.
Ideal case (no failures):
T06 (needs 2 steps): Agent used 2 → efficiency 1.0
T12 (needs 2 steps): Agent used 2 → efficiency 1.0
Failure cases (fewer steps, but answer quality dropped):
T04 (needs 2 steps): Agent used 0 → efficiency 0.0
Step efficiency below 1.0 has two causes:
- The Agent found a genuinely more efficient path (good)
- The Agent skipped required steps (bad)
The number alone can't distinguish them. Always read step efficiency alongside tool name accuracy.
Finding 3: Multi-Step and Edge Cases Score Lower
Simple (1 tool): tool_acc=75% trajectory=3.75/5
Multi-step (2 tools): tool_acc=67% trajectory=3.67/5
Edge cases: tool_acc=67% trajectory=3.67/5
Multi-step failures: the Agent assumed it could answer in one step, skipping intermediate lookups. Edge case failures (cancelled order): the Agent directly inferred "cancelled orders have no refund" without first querying to confirm the order status.
This distribution matches the pattern in most Agent systems: simple direct tasks score well, compound and boundary tasks score lower. A test set built only from simple cases inflates accuracy numbers and hides real problems.
Implementation
Tool Name Accuracy
def tool_name_accuracy(actual: list[str], expected: list[str]) -> bool:
"""Check if the tool call sequence exactly matches."""
if len(actual) != len(expected):
return False
return all(a == e for a, e in zip(actual, expected))
Parameter Accuracy
Flexible matching to avoid false failures from formatting differences:
def param_match_score(actual_calls, expected_args) -> float:
total, correct = 0, 0
for actual, expected in zip(actual_calls, expected_args):
for key, expected_val in expected.items():
total += 1
actual_val = actual["args"].get(key)
if actual_val is None:
continue
# Numeric tolerance (within 0.01)
if isinstance(expected_val, (int, float)):
if abs(float(actual_val) - float(expected_val)) < 0.01:
correct += 1
# Substring match for strings
elif str(expected_val).lower() in str(actual_val).lower():
correct += 1
return correct / total if total > 0 else 1.0
Trajectory Quality (LLM-as-Judge)
TRAJECTORY_JUDGE = """Evaluate this Agent trajectory quality (1-5):
User question: {question}
Tools called: {tool_calls}
Final answer: {answer}
5: All tool calls justified, each step necessary, answer complete
4: Generally sound, one minor issue
3: Right direction, but has redundancy or omissions
2: Some calls wrong, answer quality poor
1: Calls completely wrong or irrelevant
Return integer 1-5 only."""
Test Set Design
Include ground truth tool sequences, not just expected answers:
TestCase(
id="T04",
question="How much refund for order ORD-004?",
expected_tools=["get_order_status", "calculate_refund"],
expected_args=[
{"order_id": "ORD-004"},
{"amount": 99.0, "days_since_purchase": 15}
],
expected_min_steps=2,
)
Cover three difficulty levels:
Simple (single tool): basic tool recognition, ~50% of set
Multi-step: planning capability, ~30% of set
Edge cases: robustness (missing resources, empty results), ~20%
Include "no tool needed" cases:
Greetings and small talk test whether the Agent over-calls tools. A correct Agent should know "hi there" doesn't require a database query.
Summary
- Tool name accuracy 73%, parameter accuracy 100%: all failures are "didn't call when it should have" — not "called the wrong tool" — pointing directly at the tool triggering logic as the fix target
- Step efficiency below 1.0 needs context: combine it with tool name accuracy; low efficiency from skipped steps is a problem, from genuinely shorter paths is a feature
- Test sets must include multi-step and edge cases: simple single-tool tasks score 75%; multi-step and edge cases score 67%; a test set of only simple tasks would report 85%+ and hide the real failure modes
References
- BFCL — Berkeley Function Calling Leaderboard
- Full demo code: eval-05-agent
Check out PrimeSkills — a curated marketplace of AI agents and skills that have been validated in real-world, enterprise-grade workflows. No fluff, just what actually works.
Find more useful knowledge and interesting products on my Homepage
Top comments (0)