DEV Community

Lily
Lily

Posted on Originally published at dev.to

7 of My 8 Claude Code Agents Had Zero Calls in 30 Days: Finding Dead Agents Automatically

I had eight custom agents defined in Claude Code. When I finally counted, seven of them hadn't been called once in the last 30 days. What keeps my ¥1.2M/month automation setup running isn't clever prompting. It's an environment that keeps checking, automatically, whether the things I built are actually doing anything.

Why this setup works

Claude Code lets you define custom agents by dropping .md files into the ~/.claude/agents/ directory. You define specialists like architect (architecture design), code-reviewer (code review), and security-reviewer (security audits), and expect Claude Code to pick the right one on its own. It's a natural assumption.

But when you actually tally the logs, the results are surprising.

Take my environment as an example. ~/.claude/agents/ currently holds eight agent definition files.

architect.md
code-reviewer.md
database-reviewer.md
INDEX.md
planner.md
python-reviewer.md
security-reviewer.md
typescript-reviewer.md
Enter fullscreen mode Exit fullscreen mode

~/.claude/logs/agent-invocations.jsonl holds 682 records spanning May 28 to August 30, 2026. Aggregating the last 30 days gives this breakdown:

=== Agent usage (last 30d) ===
total invocations: 23  unique types: 3

Top 10:
  agent                                     calls  errors
  Explore                                      19       0
  general-purpose                               3       0
  code-reviewer                                 1       0

0-call agents (defined locally but not used in 30d): 7
  - INDEX
  - architect
  - database-reviewer
  - planner
  - python-reviewer
  - security-reviewer
  - typescript-reviewer
Enter fullscreen mode Exit fullscreen mode

Of the eight defined agents, exactly one, code-reviewer, was called even once in 30 days. The other seven had zero calls. 87.5% of the agents I'd defined might as well not have existed.

Narrow it to the last 7 days and it gets worse: code-reviewer drops out too, and the zero-call list grows to eight.

=== Agent usage (last 7d) ===
total invocations: 3  unique types: 2

0-call agents (defined locally but not used in 7d): 8
  - INDEX
  - architect
  - code-reviewer
  - database-reviewer
  - planner
  - python-reviewer
  - security-reviewer
  - typescript-reviewer
Enter fullscreen mode Exit fullscreen mode

This isn't just a "what a waste" story. Claude Code agent definitions are injected into the system prompt on every request. Open a large agent like architect.md and you'll find a definition of more than 220 lines. Seven unused agent definitions were burning tokens and quietly degrading inference quality the whole time.

The "defined = working" fallacy

The sense of accomplishment when you define an agent is real. "From now on, my code gets reviewed automatically." "When I think about architecture, an expert steps in." You believe that, and weeks go by.

In reality, unless an agent is explicitly specified, Claude picks the generic route (general-purpose) or Explore. Even if code-reviewer's description says "MUST BE USED for all code changes," that's text inside the definition. Claude doesn't autonomously read that instruction and act on it. The agent only works once there's a calling prompt or calling logic on the invoking side.

Unused agents cost you in two ways.

Token cost. The length of the agent catalog injected into the system prompt is paid on every invocation. More definition files means more tokens per request, eating into a large context window.

Cognitive cost. It's hard for a human to manually track which agents are actually functioning, and management gets more complex as definition files pile up. When definitions that don't reflect reality accumulate, trust in the environment erodes. The moment you wonder "is this agent even running?", your confidence in the autonomous setup wavers.

The fix: prune definitions based on measured numbers

The solution is to decide based on real numbers, not gut feeling. Build an operational cycle that uses logs to automatically surface "agents not called once in the last 30 days" and then delete or tidy them up.

The key idea is to invest in the environment, not the task. Not a one-off "delete this agent today," but a standing state where "a script exists that can instantly show me unused agents at any time." Run it monthly, weekly, or on a cron schedule; the cadence can be decided later. What matters is being permanently in a position to judge from measured data.

The reason I could build a ¥1.2M/month autonomous environment in six months isn't that I expected "AI to get smarter." It's that I invested mainly in mechanisms that constantly watch for "AI moving in the wrong direction." Monitoring agent usage is one example.

The overall flow

The system has three layers.

┌─────────────────────────────────────────────────────────┐
│  Layer 1: 記録                                          │
│  Claude Codeのstop hookが                               │
│  エージェント呼び出しをJSONLへ書き出す                  │
│                                                         │
│  ~/.claude/logs/agent-invocations.jsonl                 │
│  → 1行1レコード / ts・session_id・subagent_type等       │
└────────────────────┬────────────────────────────────────┘
                     │
                     ▼
┌─────────────────────────────────────────────────────────┐
│  Layer 2: 集計                                          │
│  agent-usage-summary.sh が指定期間のレコードを集計      │
│                                                         │
│  - Bash外殻(引数パース・環境変数セット)               │
│  - Python3ヒアドキュメント(ロジック本体)              │
│    ├ ウィンドウ期間でフィルタ                           │
│    ├ subagent_type別にカウント                         │
│    └ ~/.claude/agents/*.md と突き合わせ               │
└────────────────────┬────────────────────────────────────┘
                     │
                     ▼
┌─────────────────────────────────────────────────────────┐
│  Layer 3: 出力                                          │
│  Top10呼び出しランキング + 0回エージェント一覧         │
│                                                         │
│  → 削除・アーカイブ・再設計の判断材料になる             │
└─────────────────────────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

(Layer 1: Recording. Claude Code's stop hook writes agent invocations to JSONL, one record per line with ts, session_id, subagent_type, etc. Layer 2: Aggregation. agent-usage-summary.sh aggregates records for the given window, with a Bash shell for argument parsing and environment variables, and a Python3 heredoc for the logic: filter by window, count by subagent_type, cross-reference against ~/.claude/agents/*.md. Layer 3: Output. Top 10 invocation ranking plus a zero-call agent list, which feeds delete/archive/redesign decisions.)

Layer 1: JSONL recording via the stop hook

Claude Code has a stop hook that can run an arbitrary script when an agent invocation completes. I use this hook to append information about the invoked agent to a JSONL file.

An actual log record looks like this:

{"ts": "2026-08-25T01:32:02.235Z", "session_id": "d82e3fca-d397-4f40-8268-34bdeb9de46a", "cwd": "/dev/affiliate-fc2", "tool_use_id": "toolu_01HQ6HRVEgvnNjqDrmejPZ4S", "subagent_type": "general-purpose", "description": "Find CTA redirect click data for fc2 lane", "duration_ms": 3407, "status": "ok", "caller": {"type": "direct"}}
{"ts": "2026-08-30T08:55:08.361Z", "session_id": "36b40280-ff53-4f66-9582-aa09b7fbec80", "cwd": "/dev/note-autolike", "tool_use_id": "toolu_01RjC237NX1QwsWzVUMqHbvY", "subagent_type": "Explore", "description": "Survey note paid-article infra", "duration_ms": 236, "status": "ok", "caller": {"type": "direct"}}
Enter fullscreen mode Exit fullscreen mode

ts (timestamp), subagent_type (agent type), and status (ok/error) are the main fields used for aggregation. Since duration_ms is there too, you also get the elapsed time per call. My environment currently holds 682 records, three months of tracking data since the first record on May 28, 2026.

Layer 2: The structure of agent-usage-summary.sh

The aggregation script is 103 lines. A Bash outer shell takes the arguments, and the logic is written in a Python3 heredoc. Two reasons: parsing JSONL in pure Bash gets messy, and handling shell-integrated argument processing in pure Python is a hassle. The design plays to each one's strengths.

Here's the full script.

#!/usr/bin/env bash
# agent-usage-summary.sh — Stop hook が記録した agent 呼び出しを集計
#
# 使い方:
#   agent-usage-summary.sh           # デフォルト 7d
#   agent-usage-summary.sh 30d       # 30日
#   agent-usage-summary.sh 7d 30d    # 両方

set -uo pipefail

LOG="$HOME/.claude/logs/agent-invocations.jsonl"
AGENTS_DIR="$HOME/.claude/agents"

WINDOWS=("$@")
if [ ${#WINDOWS[@]} -eq 0 ]; then
    WINDOWS=("7d")
fi

if [ ! -f "$LOG" ]; then
    echo "no log yet: $LOG"
    exit 0
fi

export LOG_PATH="$LOG"
export AGENTS_DIR_PATH="$AGENTS_DIR"
export WINDOWS_CSV="$(IFS=,; echo "${WINDOWS[*]}")"

python3 - <<'PY'
import os, json, datetime, glob, sys
from collections import Counter

log_path = os.environ["LOG_PATH"]
agents_dir = os.environ["AGENTS_DIR_PATH"]
windows = os.environ["WINDOWS_CSV"].split(",")

def parse_window(s):
    s = s.strip().lower()
    if s.endswith("d"):
        return datetime.timedelta(days=int(s[:-1]))
    if s.endswith("h"):
        return datetime.timedelta(hours=int(s[:-1]))
    raise ValueError(f"bad window: {s}")

now = datetime.datetime.now(datetime.timezone.utc)

records = []
with open(log_path, "r", encoding="utf-8", errors="replace") as f:
    for line in f:
        try:
            r = json.loads(line)
        except Exception:
            continue
        ts = r.get("ts", "")
        try:
            dt = datetime.datetime.fromisoformat(ts.replace("Z", "+00:00"))
            if dt.tzinfo is None:
                dt = dt.replace(tzinfo=datetime.timezone.utc)
        except Exception:
            continue
        r["_dt"] = dt
        records.append(r)

# 既知 agent 一覧(ローカル定義の md ファイル名から推定)
known_agents = set()
if os.path.isdir(agents_dir):
    for fp in glob.glob(os.path.join(agents_dir, "*.md")):
        known_agents.add(os.path.splitext(os.path.basename(fp))[0])

for w in windows:
    try:
        td = parse_window(w)
    except Exception as e:
        print(f"[skip {w}]: {e}")
        continue
    cutoff = now - td
    recent = [r for r in records if r["_dt"] >= cutoff]
    counts = Counter(r.get("subagent_type", "") for r in recent if r.get("subagent_type"))
    errors = Counter(r.get("subagent_type", "") for r in recent if r.get("status") == "error")

    print(f"\n=== Agent usage (last {w}) ===")
    print(f"total invocations: {len(recent)}  unique types: {len(counts)}")
    if counts:
        print("\nTop 10:")
        print(f"  {'agent':<40} {'calls':>6}  {'errors':>6}")
        for name, n in counts.most_common(10):
            err = errors.get(name, 0)
            print(f"  {name:<40} {n:>6}  {err:>6}")

    if known_agents:
        used = set(counts.keys())
        unused = sorted(known_agents - used)
        print(f"\n0-call agents (defined locally but not used in {w}): {len(unused)}")
        for name in unused[:30]:
            print(f"  - {name}")
        if len(unused) > 30:
            print(f"  ... and {len(unused) - 30} more")
    else:
        print(f"\n(no local agents dir at {agents_dir}; cannot list 0-call agents)")
PY
Enter fullscreen mode Exit fullscreen mode

Three design points.

Multiple windows can be compared in one command. Run agent-usage-summary.sh 7d 30d and the 7-day and 30-day results print back to back. Trend shifts like "used in the last 30 days but zero in the last 7" are visible at a glance.

Known agents are cross-referenced via glob. Filenames under ~/.claude/agents/*.md with the extension stripped are treated as "defined agents." Add a new agent and it's automatically picked up without touching the script.

Error counts are output at the same time. Records with status: "error" are tallied separately, so agents that are "called but failing every time" become visible too. Tracking success rate, not just call count, lets you catch a different class of problem: "running but broken."

Layer 3: Reading the output and deciding what to do

The script's output has two blocks: the Top 10 ranking and the 0-call list.

=== Agent usage (last 30d) ===
total invocations: 23  unique types: 3

Top 10:
  agent                                     calls  errors
  Explore                                      19       0
  general-purpose                               3       0
  code-reviewer                                 1       0

0-call agents (defined locally but not used in 30d): 7
  - INDEX
  - architect
  - database-reviewer
  - planner
  - python-reviewer
  - security-reviewer
  - typescript-reviewer
Enter fullscreen mode Exit fullscreen mode

What the ranking tells you is simple. Of 23 invocations, 19 (82.6%) are Explore. Explore is a general-purpose agent specialized in file search and code investigation, a built-in Claude Code feature rather than a custom definition. In other words, the numbers confirm that "I defined seven custom agents, but in practice only the generic built-ins were used" had been going on for three months.

For each agent that appears on the 0-call list, there are three choices.

Delete it. If it's clearly unused and no calling mechanism has been built, delete it. You immediately save system prompt tokens and make the environment easier to reason about.

Archive it. If it might be useful in the future but isn't needed now, move it to ~/.claude/agents/archive/. Because the glob pattern is restricted to *.md, moving a file into a subdirectory automatically drops it from the tally.

Build a caller. If the agent's functionality is genuinely valuable and it's simply "not being called," add a mechanism that explicitly invokes it, via a stop hook or a specific prompt pattern. Even here, without numbers you're just "assuming it's valuable," so re-run the tally after implementing and confirm the effect.

Implementation details

The first half gave the big picture. Now I'll read through the code of the two scripts, stop_agent_tracker.sh (recording) and agent-usage-summary.sh (aggregation), focusing on "why it's written this way" and "where the crux is."

stop_agent_tracker.sh: the two-pass structure is the core

What the stop hook receives is JSON like the following, which Claude Code streams to standard input at session end:

{"session_id":"36b40280-...","transcript_path":"/.../.claude/projects/.../transcript.jsonl","cwd":"/dev/note-autolike","hook_event_name":"Stop"}
Enter fullscreen mode Exit fullscreen mode

transcript_path points to the full conversation log for that session. Which agents were called is recorded there. But a single agent invocation is recorded as two separate lines: the tool_use at call time (type and arguments) and the tool_result after completion (success/failure and output). Only by matching those two lines do you learn "what, when, and did it succeed."

That's why the script uses two-pass processing.

# 第1パス: 全 tool_use と tool_result をインデックス化
uses   = {}   # tool_use_id -> (ts, name, input, caller)
results = {}  # tool_use_id -> (ts, is_error)

with open(tp, "r", encoding="utf-8", errors="replace") as f:
    for line in f:
        rec = json.loads(line)
        for b in content:
            if btype == "tool_use" and b.get("name") == "Agent":
                inp = b.get("input") or {}
                if "subagent_type" not in inp:
                    continue
                uses[uid] = (ts, b.get("name"), inp, b.get("caller"))
            elif btype == "tool_result":
                results[rid] = (ts, bool(b.get("is_error")))
Enter fullscreen mode Exit fullscreen mode

The first pass scans every line of the transcript and accumulates uses and results in separate dictionaries. The second pass iterates over uses, looks up the matching entry in results, and writes out JSONL.

There's one important design decision here. Unmatched tool_uses (no key in results) are recorded with status: "pending".

res = results.get(uid)
if res:
    res_ts, is_error = res
    status = "error" if is_error else "ok"
else:
    res_ts, status = None, "pending"
Enter fullscreen mode Exit fullscreen mode

Claude Code fires the hook at session end. If a session was force-killed midway, or the session dropped before the tool_result was written, the record stays pending. This lets you distinguish "recorded but never completed" calls.

Computing duration_ms is another benefit of the two-pass structure.

t0 = parse_ts(use_ts)   # tool_use のタイムスタンプ
t1 = parse_ts(res_ts)   # tool_result のタイムスタンプ
if t0 and t1:
    duration_ms = int((t1 - t0).total_seconds() * 1000)
Enter fullscreen mode Exit fullscreen mode

The difference between the tool_use and tool_result timestamps is the agent's execution time. Looking at the real logs, Explore takes 236ms while general-purpose takes 3407ms to 6830ms. That gap roughly approximates the gap in token cost.

Deduplication logic is also worth noting. A stop hook can fire multiple times in a single session (Claude Code restarts, reconnecting after a force-kill, and so on). Writing the same tool_use_id twice would corrupt the tally.

seen_ids = set()
if os.path.exists(out_path):
    with open(out_path, "r", encoding="utf-8", errors="replace") as f:
        for line in f:
            r = json.loads(line)
            if r.get("session_id") == sid and r.get("tool_use_id"):
                seen_ids.add(r["tool_use_id"])
Enter fullscreen mode Exit fullscreen mode

Before writing, it scans every line of the JSONL and collects the tool_use_ids within the same session_id into a set. The write loop skips anything in seen_ids. The fact that 682 records contain not a single duplicate is proof this works.

agent-usage-summary.sh: why split into a Bash shell and a Python core

Looking at this design, some people will think "just write it all in Python." I thought so too at first.

I kept the Bash shell for two reasons.

First: flexible argument handling. Using "$@" lets you handle multiple arguments like 7d 30d naturally. In Python you'd have to parse sys.argv yourself, and dealing with the array is a bit more awkward.

Second: passing data via environment variables. The Python inside the heredoc (<<'PY') receives Bash-side variables through os.environ.

export LOG_PATH="$LOG"
export AGENTS_DIR_PATH="$AGENTS_DIR"
export WINDOWS_CSV="$(IFS=,; echo "${WINDOWS[*]}")"
Enter fullscreen mode Exit fullscreen mode

The WINDOWS_CSV construction, IFS=,; echo "${WINDOWS[*]}", is the key. It converts the Bash array into a comma-separated string before handing it to Python, which then .split(",")s it back. Bash arrays can't be passed directly into a heredoc, so you need this bridge that encodes them as a string first.

On the Python side, parse_window converts strings into timedeltas.

def parse_window(s):
    s = s.strip().lower()
    if s.endswith("d"):
        return datetime.timedelta(days=int(s[:-1]))
    if s.endswith("h"):
        return datetime.timedelta(hours=int(s[:-1]))
    raise ValueError(f"bad window: {s}")
Enter fullscreen mode Exit fullscreen mode

Right now it supports only d (days) and h (hours). If you want to add w (weeks) or m (months), changing just this function is enough for the whole script to work.

Automatic detection of known agents is another important piece of the design.

known_agents = set()
if os.path.isdir(agents_dir):
    for fp in glob.glob(os.path.join(agents_dir, "*.md")):
        known_agents.add(os.path.splitext(os.path.basename(fp))[0])
Enter fullscreen mode Exit fullscreen mode

It puts every *.md filename (minus the extension) directly under ~/.claude/agents/ into a set. In my environment, moving a file into the archive/ subdirectory is enough to drop that agent from the tally. No script changes required. Older agent definitions currently live in the archive/ directory (created August 29, 2024).

Where I got stuck

Here's what it took to actually get this working. Every one of these was a "it should work, so why is nothing being recorded?" problem, and it took a while to get from symptom to cause.

Snag 1: the agent tool is named "Agent," not "Task"

The code I wrote first searched the transcript for name == "Task". Claude Code's public-facing API had introduced agent invocation under the name "Task."

But when I actually opened the transcript and checked its contents, every record said "name": "Agent".

{"type": "tool_use", "name": "Agent", "input": {"subagent_type": "Explore", ...}}
Enter fullscreen mode Exit fullscreen mode

Because I was filtering on name == "Task", every record was skipped and nothing was recorded for two days. I only noticed after checking directly with grep '"name"' ~/.claude/projects/*/transcript.jsonl | head -5. A gap between the documentation and the actual file.

The fix was one line.

# 修正前
if btype == "tool_use" and b.get("name") == "Task":
# 修正後
if btype == "tool_use" and b.get("name") == "Agent":
Enter fullscreen mode Exit fullscreen mode

The lesson: don't trust the docs, read the actual file. transcript.jsonl is ordinary JSONL, so you can always inspect it directly.

Snag 2: subagent_type lives at input.subagent_type

This one came from not pinning down the exact structure of tool_use. At first I tried to fetch it with b.get("subagent_type"). That always returns None.

Re-checking an actual transcript record, it looks like this:

{
  "type": "tool_use",
  "id": "toolu_01RjC237NX1QwsWzVUMqHbvY",
  "name": "Agent",
  "input": {
    "subagent_type": "Explore",
    "description": "Survey note paid-article infra",
    "prompt": "..."
  }
}
Enter fullscreen mode Exit fullscreen mode

subagent_type is inside input. The correct access is b.get("input", {}).get("subagent_type"). The current code pulls input out first with inp = b.get("input") or {} and then reads inp.get("subagent_type").

When the problem hit, the log that should have had 682 records had zero. The guard clause if "subagent_type" not in inp: continue was rejecting every single one. To debug, I set the CC_AGENT_TRACKER_DEBUG=1 environment variable to enable debug logging.

CC_AGENT_TRACKER_DEBUG=1 bash ~/.claude/hooks/stop_agent_tracker.sh <<< '...'
Enter fullscreen mode Exit fullscreen mode

stop_agent_tracker.log showed recorded=0 total_uses=0, confirming that "not a single tool_use was being recognized." So I opened the transcript's raw JSON directly, checked the structure of input, and fixed it in one line.

Snag 3: set -uo pipefail dies when the WINDOWS array is empty

set -uo pipefail turns any reference to an undefined variable into an immediate error. That's the right setting in itself, but when called without arguments, the script tried to reference ${WINDOWS[*]} before ${#WINDOWS[@]} could return 0, and errored out.

Concretely, the original code was this:

WINDOWS=("$@")
export WINDOWS_CSV="$(IFS=,; echo "${WINDOWS[*]}")"  # 空配列で問題発生
Enter fullscreen mode Exit fullscreen mode

Call it with no arguments and WINDOWS is an empty array. Under the -u flag, expanding an empty array can be an error (the behavior differs subtly between zsh and bash), so WINDOWS_CSV ends up empty or, worst case, the script exits.

The fix is to set a default before exporting WINDOWS_CSV.

WINDOWS=("$@")
if [ ${#WINDOWS[@]} -eq 0 ]; then
    WINDOWS=("7d")
fi
export WINDOWS_CSV="$(IFS=,; echo "${WINDOWS[*]}")"
Enter fullscreen mode Exit fullscreen mode

Do the empty check first, assign the default, then export. Under set -u you just follow one simple rule: always settle a variable's value before using it.

This problem only surfaced when I ran it automatically from cron. Manual runs always passed arguments, so I never noticed, but the cron definition had no arguments, so it silently failed every night. I only realized "there are nights with zero records" after checking tail -20 /var/log/....

The first record in the log has the session ID TEST-AGENT-TRACKER-001, another artifact of the debugging process. The record from when I manually fed dummy data to test whether the hook worked is still sitting at the head of the 682 entries. Mixing production logs with test data is untidy, but the aggregation logic filters by timestamp, so there's no real harm.

Snag 4: code-reviewer doesn't read the "MUST BE USED" in its own definition

This isn't a technical bug but a snag born of a fundamental misunderstanding.

The description at the top of code-reviewer.md says MUST BE USED for all code changes. When I first defined the agent, I believed "now a review runs automatically on every code change."

Yet in the actual 30-day tally, code-reviewer was called exactly once.

Re-examining how Claude selects agents: the description is "a hint for deciding which agent to pick," not "a command to force-call this agent." Unless the calling prompt or a hook specifies it explicitly, Claude takes the generic route.

Strong wording like MUST BE USED functions as a rule to be followed inside the agent once it has been selected. It has no effect on forcing activation from outside the agent.

Once you understand that, there are two options. Either (1) bake an instruction like "use code-reviewer after code changes" into a stop hook or prompt template, or (2) "if it's never going to be used, delete it."

I'm currently choosing (2). code-reviewer was called once in 30 days, and that one time was when I specified it manually. Without a mechanism, the call count won't rise. I judged that the cost of a 323-line definition file permanently occupying the system prompt outweighed the expected value of "might use it someday."

I could make that call only because I had the numbers. Without the fact of "once in 30 days," the vague hope of "maybe it's working" would have lingered forever.


To sum up the implementation and the failures.

The crux of this system, which is complete in two scripts, is maintaining a state where "real numbers come out whenever I want to check." With numbers you can decide. Without them you keep running on hope, and wasted tokens and system prompt bloat quietly pile up.

Build the mechanism that measures whether things get called before you add more definition files. That ordering is the basic posture for growing Claude Code into an autonomous environment.

Pitfalls

The earlier sections covered four snags. Here I'll list additional pitfalls, grouped by "environment-specific," "operational phase," and "misinterpretation." Only ones I actually stepped on.


PATH is dead when running from cron.

It works when run manually, but fails with python3: command not found when run from a cron definition. The cron execution environment doesn't load ~/.zshrc or ~/.profile, and PATH is roughly /usr/bin:/bin. /usr/local/bin/python3 and node under nvm become invisible. Two remedies: hardcode PATH at the top of the script, or write PATH=/usr/local/bin:/usr/bin:/bin on the first line of the cron definition. I use the latter. Since HOME may also be unset, you need to write the absolute path /Users/youraccount/ in the cron entry instead of ~/ (the $HOME inside the script itself is fine as long as the HOME environment variable is set).

Don't let one broken JSONL line stop the whole script.

With 682 accumulated records, an incomplete JSON line can sneak in. If the session drops while Claude Code is invoking the hook, the last record can end up half-written. If you write json.loads without a try/except, a single corrupt line kills the whole script. The aggregation script's current code skips with except Exception: continue, ignoring the broken line and processing the rest. Skip the try/except because "that case will never happen," and it'll fail for the first time three months later once the log has grown.

glob("*.md") picking up the archive directory.

glob.glob(os.path.join(agents_dir, "*.md")) targets only .md files directly under ~/.claude/agents/. Files moved into the archive/ subdirectory are not included, which is intended. But if you rewrite it as glob.glob(os.path.join(agents_dir, "**/*.md"), recursive=True), archived agents are treated as "defined" again and reappear on the 0-call list. You get an "I archived it but it hasn't gone away" situation. The correct answer is not to add recursive=True.

INDEX.md is misdetected as an agent.

My ~/.claude/agents/INDEX.md is not an agent definition but an index file for the directory. Since glob("*.md") looks only at the extension, the name INDEX gets counted as a "defined agent." As a result, INDEX always shows up on the 0-call list. There's no real harm, but every time I see the list I pay the cost of thinking "what was this again?" Remedies: move the INDEX file to a subdirectory, keep an exclusion list in the script, or don't put such files there in the first place. For now I've left it and mentally filed "INDEX at zero is normal."

It's not obvious why Explore and general-purpose never appear on the 0-call list.

The 0-call list only shows "things with a file in ~/.claude/agents/ that weren't called." Explore and general-purpose are Claude Code built-in agents with no local .md file. They aren't in known_agents, so they never appear on the 0-call list. That's by design, but it confused me at first: "why doesn't Explore show up?" You need to read the list knowing that built-in agents dominate the top of the Top 10.

There are cases where the stop hook doesn't run.

The stop hook fires on normal session termination. It doesn't fire on Ctrl+C force-quits, process kills, or Claude Code crashes. So you get "that long session wasn't recorded." I can't tell how many of the 682 are missing, but I operate on the premise that "I can analyze what was recorded." Demanding perfect records stops operations.

pending status records get mixed into the tally.

Records with status: "pending" were recorded mid-session. The aggregation script doesn't filter by status and counts everything. That means "started but unknown whether completed" calls are included in the call count. Checking the actual breakdown of the 682 records, pending is a tiny minority, but if you want more precision you need to add a "status" != "pending" condition. Current policy is "tolerate a little error."

Misreading the error rate.

The errors column in the Top 10 is all zeros, but that doesn't mean no errors ever occurred. Only records with status: "error" count as errors. Outcomes like "the answer was incomplete" or "it couldn't find the file" are all recorded as ok. "Few errors = running healthily" is overstating it. Read it as "few fatal errors at the recording level."

cwd values get mixed across multiple projects.

Each JSONL record includes cwd. My log has /dev/affiliate-fc2, /dev/note-autolike, /dev/... all mixed together. The current aggregation script doesn't distinguish projects and sums everything. If you want analysis like "project A uses Explore heavily but project B never does," you need to add an option to filter on the cwd field. For now the company-wide total is enough, so it's unimplemented, but I plan to add it as the number of projects grows.

Underestimating the size of definition files.

It's easy to shrug off 0-call agents with "eh, whatever," but the numbers change your view. The current eight files total 109,852 bytes (about 107KB) and 1,221 lines. The big ones are code-reviewer.md (323 lines), planner.md (221 lines), and architect.md (220 lines). These get injected into the system prompt on every request. How much 107KB matters depends on how you use the overall context window, but you really do feel "responses got faster after deleting unused definitions." Putting a number on it is what finally gets you moving.


Best practices

Here's what proved effective in actual operation, along with lessons learned from failures.


1. Check the log format in the raw file before writing the script.

Run grep '"name"' ~/.claude/projects/*/transcript.jsonl | head -5 first. Whether it's "Task" or "Agent", and which level subagent_type sits at, the actual file is the truth, not the docs. This one command prevents two wasted days.

2. Don't believe "the agent I defined is working" until you've measured it.

The phrase MUST BE USED is an internal rule for after the agent has been selected. It has no effect on forced invocation from outside the agent. Right after defining, run agent-usage-summary.sh 7d, and if it's still at zero a week later, decide immediately: build a caller or delete it.

3. Always show the 7-day and 30-day windows side by side.

The agent-usage-summary.sh 7d 30d combination shows both "recently started using" and "used before but not lately" in one command. An agent called just once in 30 days shows zero in the 7-day tally. That gap tells you it was "a one-off, incidental use."

4. Don't be afraid to delete. Use the archive selectively.

The "might use it someday" thought pattern preserves unneeded agents. Keep the criterion simple: "zero calls in 30 days and no calling mechanism exists means delete; otherwise archive." Archiving is just moving to ~/.claude/agents/archive/, and you can restore it if needed. Since glob only looks at the top level, it drops out of the tally the moment you move it.

5. Re-run the tally right after deleting to confirm the effect.

After deleting an agent, run agent-usage-summary.sh 30d and confirm the 0-call list got shorter. If "I deleted it but it's still there," the file remains or there's a copy elsewhere. The habit of comparing numbers before and after each change keeps the environment trustworthy.

6. Make the stop hook's records manually checkable after a session ends.

tail -5 ~/.claude/logs/agent-invocations.jsonl | python3 -m json.tool lets you check the latest five records any time. The habit of confirming "was today's session recorded?" doubles as a liveness check on the hook. No records for a week is a sign the hook is broken.

7. If the error count suddenly rises, investigate it first.

In normal times the error count is zero. If a number appears in the errors column of the Top 10 table, that agent is having problems. Extract the records with grep '"status":"error"' ~/.claude/logs/agent-invocations.jsonl, then trace the transcript from the session_id to find out what happened. Monitoring the error rate matters as much as monitoring usage.

8. Tally pending status separately to understand the loss rate.

Periodically check grep '"status":"pending"' ~/.claude/logs/agent-invocations.jsonl | wc -l. If the pending count among the 682 is trending up, either there are many force-quits (unstable sessions) or the hook is dying partway through. Under 10% loss is acceptable; above that, start investigating.

9. Explicitly set HOME and PATH in cron definitions.

0 9 * * 1 HOME=/Users/自分/ PATH=/usr/local/bin:/usr/bin:/bin bash ~/claude/scripts/agent-usage-summary.sh 7d 30d >> ~/agent-weekly.log 2>&1
Enter fullscreen mode Exit fullscreen mode

HOME can't expand $HOME, so hardcode the value. PATH must include wherever python3 is visible. Redirecting output to >> ~/agent-weekly.log tells you why it failed when it does.

10. Keep the tallies as a weekly report.

Add agent-usage-summary.sh 7d 30d >> ~/.claude/logs/agent-weekly-report.log to a weekly cron and accumulate the log. A change like "two months ago python-reviewer was called five times a month, now it's zero" can reflect library changes or a shift in the kind of work. Being able to track changes over time lets you review your agent design.

11. Don't define an agent before building the calling logic.

"Define first, think about callers later" is the root of the problem. When you define an agent, decide at the same time "when and how will this be called?" A hook, a prompt template, automatic execution after a specific command. Unless one of those exists, the definition file only bloats the system prompt.

12. Check existing agents' call counts before adding a new one.

Run agent-usage-summary.sh 30d before adding, and if the 0-call list is long, tidy up first. Maintain an environment where "only the things being used exist," not "the old ones stopped being used after I added new ones." The smaller the total volume of definition files, the clearer Claude Code's agent selection becomes.

13. Use duration_ms to identify heavy agents.

Explore averages 236ms; general-purpose runs 3,407ms to 6,830ms. If a heavy agent is called frequently, it's worth considering whether Explore could cover that use case instead. Since duration_ms is recorded in the JSONL, you can compute mean and median with python3 -c "import json,statistics; data=[json.loads(l) for l in open('~/.claude/logs/agent-invocations.jsonl')]; ...".


Summary

The "defined = working" fallacy produces the quietest cost of all when growing a Claude Code environment.

In this measurement, of eight agent definitions, only one type was called in 30 days, and only once. The remaining seven, a combined 1,100 lines and roughly 95KB of definitions, kept occupying the system prompt on every request. Two scripts made this visible: stop_agent_tracker.sh records agent invocations to JSONL, and agent-usage-summary.sh aggregates by window and agent and outputs the 0-call list.

Three key points about the mechanism.

Two-pass processing of transcript.jsonl. tool_use and tool_result are recorded on separate lines. Only by matching them do you get "what, when, and did it succeed" together. Write it as a single pass and you only get one side, leaving incomplete records.

Division of labor between a Bash shell and a Python core. Leave argument handling and environment variable passing to Bash, and JSONL parsing and aggregation to Python. Try to write it in just one and you'll get stuck on the part that language isn't good at.

Cross-referencing against ~/.claude/agents/*.md. The 0-call list the tally produces can't be built from logs alone. Detecting "defined but not called" requires information from the filesystem side. That single step of grabbing filenames via glob and taking the set difference is the heart of this script.

Deleting isn't scary. Moving to the archive drops it from the tally, and you can bring it back if needed. "Without numbers you can't decide; with numbers you don't hesitate." That's the basic posture for growing an autonomous environment.

What supports ¥1.2M/month isn't the intelligence of the AI. It's an environment where you can measure whether things are running. Because you can measure, you can cut. Because you can cut, the AI can focus on its real work. Two scripts keep that cycle turning.

How many of your own custom agents would survive a 30-day zero-call check?


The full picture of the system, the breakdown of the ¥1.2M/month, and the 30-day procedure are compiled in a paid note.
📕 How to actually earn with a Claude Code autonomous environment: the system, real examples, getting started, and support


Written by **Lily* — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*

Top comments (0)