When evaluating AI agents, we often focus on the final answer.
Was it correct?
Was it useful?
Was it grounded?
That matters.
But for agents, there is another important question:
How did the agent get there?
This is where agent trajectory comes in.
What is an agent trajectory?
Agent trajectory means the path an agent takes before producing the final answer.
It includes the steps the agent went through, such as:
router decisions
tool calls
database lookups
retrieval steps
LLM calls
repeated actions
final response
A simple agent path may look like this:
User input
→ Router
→ Database lookup
→ Data analyzer
→ Output
A messy agent path may look like this:
User input
→ Router
→ Database lookup
→ Router
→ Data analyzer
→ Router
→ Database lookup
→ Router
→ Data analyzer
→ Output
Both paths may eventually produce the correct answer.
But they are not equally good.
The second path takes more steps, costs more, takes longer, and creates more chances for something to fail.
Why trajectory matters
Imagine a user asks:
“Show me last month’s sales trend.”
The agent may need to:
- Understand the question
- Choose the right database tool
- Fetch the data
- Analyze the result
- Generate the final answer
That is a reasonable path.
But an inefficient agent may:
- call the same tool multiple times
- keep going back to the router
- repeat the same analysis
- use the wrong tool first
- query the database again without need
- take many steps before giving the answer
The final answer may look okay.
But the system behavior is not okay.
In production, inefficient paths matter because they increase:
- latency
- cost
- API usage
- tool calls
- failure risk
- debugging complexity
- user wait time
This is why agent evals should evaluate both:
Output quality
+
Path quality
What is convergence?
Convergence measures how closely an agent follows the optimal path for a given query.
Simple meaning:
Is the agent reaching the answer cleanly, or is it wandering around?
If the agent takes the shortest useful path most of the time, convergence is high.
If it keeps taking unnecessary steps, convergence is lower.
A simple convergence score
One way to calculate convergence is by running the agent on similar queries and recording how many steps it takes.
Let:
N = total number of runs
S_agent,i = number of steps taken by the agent in run i
S_optimal = shortest successful path length across the runs
Then:
Convergence score =
average of min(1, S_optimal / S_agent,i)
Or written more formally:
Overall Convergence Score = (Σᵢ₌₁ᴺ min(1, S_optimal / S_agent,i)) / N
Simple example:
Assume the shortest successful path takes 5 steps.
If one agent run takes 5 steps:
score = min(1, 5 / 5) = 1
That is ideal.
If another run takes 10 steps:
score = min(1, 5 / 10) = 0.5
If another run takes 20 steps:
score = min(1, 5 / 20) = 0.25
Lower score means the agent is taking more steps than needed.
What a convergence score tells us
A convergence score helps answer:
- Is the agent taking the shortest useful path?
- Is it repeating unnecessary steps?
- Is it using tools efficiently?
- Is it getting stuck before answering?
- Is it consistent across similar inputs?
This is useful because two agents may both answer correctly, but one may do it in a much cleaner way.
But convergence is not enough
Convergence should not be measured alone.
A short wrong path is not better than a longer correct path.
For example, if the agent gives a wrong answer in 3 steps, that is not good just because it was fast.
So convergence should be evaluated together with:
- final answer correctness
- task success
- tool choice accuracy
- grounding
- safety
- output quality Efficiency only matters after the agent is doing the right thing.
How this connects to traces
To measure trajectory and convergence, we need traces.
A trace shows what happened during an agent run.
It can show:
which steps happened
how many steps were taken
which tools were called
whether steps repeated
whether the agent made progress
where time was spent
Without traces, we only see the final answer.
With traces, we can inspect the path.
That is why observability and evals work well together.
Observability shows what happened.
Evals help decide whether it was good or bad.
Final thought
Agent evals are not only about final answers.
They are also about behavior.
A good agent should answer correctly.
But it should also take a clean, efficient, and understandable path.
That is why trajectory matters.
And that is why convergence can be a useful signal when evaluating AI agents.
Top comments (0)