DEV Community

Mukunda Rao Katta
Mukunda Rao Katta

Posted on

After every Hermes agent run, I print a 10-line summary. Now I know exactly what happened.

#hermesagentchallenge #devchallenge #agents #python

Hermes Agent Challenge Submission: Build With Hermes Agent

This is a submission for the Hermes Agent Challenge.

After a long Hermes agent run, I had a 3,000-line JSONL trace. I could open it and read it, but I had no quick way to answer: how much did that cost? How many tool calls? Were there errors? How long did it take?

trace-summary answers those questions in one function call.

One call, readable output 

from trace_summary import summarize_file

s = summarize_file("logs/run.jsonl")
print(s)
Enter fullscreen mode Exit fullscreen mode 
Events:        142
Errors:        3
Cost:          $0.0847
Tokens:        48,200 (in=32,100 out=16,100)
Duration:      183.4s
Lanes:         supervisor, worker-1, worker-2
Event kinds:   tool_call=89, llm_call=42, log=11
Tool calls:    web_search×45, read_file×28, arxiv_search×16
Enter fullscreen mode Exit fullscreen mode

That's the whole story. I print this at the end of every run and log it to Slack. In 10 lines I know whether the run succeeded, what it cost, which tools dominated, and whether there were errors.

Works on any JSONL format

The library auto-detects field names used across common agent frameworks:

What Recognized keys
Cost cost_usd, cost, price_usd, usd
Tokens in tokens_in, input_tokens, prompt_tokens
Tokens out tokens_out, output_tokens, completion_tokens
Timestamp timestamp, ts, time, created_at, at
Tool name tool, tool_name, name, step
Error error, err, exception
Event kind kind, type, event_type
Lane lane

If your JSONL uses any of those keys, the summary just works.

From a list instead of a file 

from trace_summary import summarize_trace

# After collecting events in memory
s = summarize_trace(events)
logger.info(
    "Run complete",
    extra={
        "cost_usd": s.total_cost_usd,
        "errors": s.error_count,
        "duration_s": s.duration_seconds,
    }
)
Enter fullscreen mode Exit fullscreen mode

The TraceSummary object 

@dataclass
class TraceSummary:
    event_count: int
    error_count: int
    total_cost_usd: float
    total_tokens_in: int
    total_tokens_out: int
    first_ts: float | None
    last_ts: float | None
    duration_seconds: float | None
    tool_counts: dict[str, int]
    kind_counts: dict[str, int]
    lanes: set[str]
Enter fullscreen mode Exit fullscreen mode

All fields are accessible for programmatic use. The render() method formats them for display.

Alert on errors or cost 

s = summarize_file("run.jsonl")

if s.error_count > 0:
    alert(f"{s.error_count} errors in agent run")

if s.total_cost_usd > 1.00:
    alert(f"Run cost ${s.total_cost_usd:.2f} — over $1 budget")
Enter fullscreen mode Exit fullscreen mode

CLI 

trace-summary run.jsonl
Enter fullscreen mode Exit fullscreen mode

Same output as print(s). Pipe it to a log, a Slack webhook, or a monitoring system.

My Hermes agent integration 

import logging
from trace_summary import summarize_trace

class HermesAgentLoop:
    def run(self, ...):
        events = []
        ...
        # collect events throughout the run
        ...
        s = summarize_trace(events)
        logging.info("Run complete:\n%s", s.render())
        if s.error_count > 0:
            raise RuntimeError(f"{s.error_count} tool errors — check trace")
Enter fullscreen mode Exit fullscreen mode

I keep one JSONL file per run, append events as they happen, and summarize at the end. The trace stays in case I need to debug. The summary is what I actually read.

Zero dependencies

Standard library only: json, pathlib, dataclasses. No third-party packages.

pip install trace-summary
Enter fullscreen mode Exit fullscreen mode

Repo: https://github.com/MukundaKatta/trace-summary

Top comments (0)