AI agent tool schemas cut 47%: the duplicate YAML key that disabled nothing

My self-hosted AI agents were slow, and I spent a while assuming the model was the problem. It wasn't. Most of every request was the agent describing its own tools to the model, over and over, for tools it never used.

On 2026-08-15 I trimmed the tool schemas on five specialist agent profiles by 47%. The trim itself was a few lines of YAML. The part that cost me the most time was a duplicate key that made the change look applied while disabling nothing at all.

Where the time was going

I run Hermes Agent with several profiles — a coder, a reviewer, a scout, a security agent and a writer, plus a default "chief of staff" profile — against a local model on a machine with a 6GB GPU.

Turns were taking a minute and a half or more, and the obvious suspect was the model. I measured a few alternatives and none of them helped: the bigger candidate couldn't fit the 6GB card at any context size, and the closest alternative I benchmarked tied on the part that mattered.

The part that mattered was prefill: processing the prompt before a single token comes back. I captured a real turn and it was about 18,400 prompt tokens. 13,541 of them, 74%, were the JSON schemas for the 27 tools the agent had enabled. Measured another way, tool schemas were 58–67% of each profile's fixed prompt. At the roughly 600 tokens per second my box manages for prefill, that is a big slice of every single turn, spent before the model has thought about anything.

So the lever wasn't the model. It was the tool list.

Measure first: hermes prompt-size --json

Hermes has an offline command that breaks the fixed prompt down: system prompt, skills index, memory, the tool count and total schema bytes, and a per-toolset breakdown. Run it per profile:

hermes --profile coder prompt-size --json

One trap: the per-toolset array is called toolsets_breakdown, not toolsets.

To get real token counts rather than guesses, I sent the text to Ollama's /api/generate and read prompt_eval_count from the response. For these schemas it came out at about 3.97 bytes per token, which makes the byte counts from prompt-size easy to convert.

I also used a wire-capture rig I had built earlier to debug a different problem: a fake OpenAI-compatible endpoint the agent is pointed at, which records exactly what goes over the wire. Two lessons from using it: keep the largest request of a run, not the first (the first call is often a small one without tools), and tag each capture from a file per request rather than from the server's environment.

Decide by usage, not by guessing

I didn't want to trim tools on a hunch, so I checked what the agents actually called. Hermes keeps each profile's message history in a local SQLite database, so I counted tool calls across roughly 1,900 messages.

• The 10 browser_* tools, text_to_speech, clarify and process had been called zero times.
• session_search and delegate_task were used, but only by the default profile (5 and 21 times).

That gave a clear, evidence-based cut:

profile	disabled toolsets
default	tts
coder, reviewer, scout, sec, writer	browser, tts, clarify, session_search, delegation

One thing I checked before touching production: the browser toolset is a 13-tool bundle that includes web_search, which the agents do use. On the capture rig, disabling browser removed only the 10 browser_* tools and left web_search alone, because core tools are protected. The bundle listing alone would have told me otherwise, so I tested it.

The trap: a duplicate YAML key

The setting is agent.disabled_toolsets in each profile's config. I added it straight after the agent: line:

agent:
  disabled_toolsets: [browser, tts, clarify, session_search, delegation]
  # ... lots of other settings ...
  disabled_toolsets: []

It looked right. grep -A6 showed my new list sitting exactly where I expected. And it did nothing.

Further down the same agent: block, the file already had disabled_toolsets: []. YAML doesn't error on a duplicate key in a mapping; PyYAML simply keeps the last one. So my list parsed to an empty list, every tool stayed enabled, and nothing complained.

The fix is boring but important:

1. Count the key before adding it: grep -c disabled_toolsets on the file. If it's already there, edit the existing line.
2. Verify with a real parse, never with grep:

python3 -c "import yaml; print(yaml.safe_load(open('config.yaml'))['agent']['disabled_toolsets'])"

grep shows you text. Only a parser shows you what the program will actually see.

Later, when I wrote a helper script to edit these profile configs, I made it load YAML with a loader that refuses duplicate keys, back up first, and re-read the file afterwards or fail. A duplicate key silently winning is too easy to miss by eye.

The second trap: hermes config set can't write lists

The obvious alternative to hand-editing was the CLI:

hermes config set agent.disabled_toolsets '["browser","tts"]'

That doesn't work either. It writes the value as a quoted string:

disabled_toolsets: '["browser","tts"]'

The filter expects a list, silently ignores the string, and tts was still present afterwards when I checked. List values have to be written as real YAML sequences by editing the file.

Both traps have the same shape: the config looks set, the tool reports no error, and the behaviour doesn't change. Only measuring the result catches it.

The result

From prompt-size, before and after:

profile	tools	tool schema bytes	fixed prompt total
specialists (×5)	27 → 13	46,403 → 24,795 (−47%)	80,057 → 58,262 (−27%)
default	24 → 23	50,058 → 48,188 (−3.7%)	74,699 → 72,829

That is about 5,400 tokens saved on every specialist turn, roughly 9 seconds at the measured prefill rate.

After the change I checked the things that could have broken: the gateway came back up and running, the coder ran a real terminal command correctly, and it could still reach its MCP servers — the small tool_search / tool_call / tool_describe trio that fronts them survived the trim. The default profile kept web search and delegation. Every file was backed up first, so reverting is a restore and a gateway restart.

What's left

Five memory-integration tools on the default profile, 7,663 bytes, have never been called. They don't appear under a normal toolset because the memory plugin injects them directly, so disabled_toolsets can't reach them. Removing them means switching that integration off, which is a product decision rather than a trim.

The remaining big toolsets — file, skills, terminal, memory — are all genuinely used. The other half of the fixed prompt is now the system prompt (about 21–24 KB) and the skills index (about 9 KB). That's the next place to look.

Takeaways

1. Measure the prompt before blaming the model. On a small GPU, prefill dominates, and tool schemas can be most of the prefill.
2. Trim by recorded usage. Message history is the evidence; zero calls in about 1,900 messages is a clear signal.
3. Never trust a config edit you verified with grep. Parse it. Duplicate YAML keys don't error; the last one wins.
4. Check how your CLI writes lists. A list quoted into a string looks fine and does nothing.
5. Measure again after every change. The only proof the trim happened was the byte count going down.

---

🤖 Drafted with AI assistance from my own homelab notes, logs and repos, then reviewed and edited before publishing.