You add a skill. It triggers. You add ten more, and the first one quietly stops firing — same file, same description, same request. Nothing errored...
For further actions, you may consider blocking this person and/or reporting abuse
Skill listing budgets are going to matter more as people add more specialized workflows. A skill that exists but is never surfaced is functionally missing.
The hard part is making descriptions do two jobs at once: precise enough to trigger at the right moment, but compact enough to survive retrieval and listing constraints. That turns skill authoring into information architecture, not just prompt writing.
"Information architecture, not just prompt writing" is a good way to put it — and those two jobs really do trade against each other, since widening a description to catch more cases tends to make it collide with a neighbour.
I gave up on judging mine by reading them and started scoring them instead — a small corpus of prompts with the skill I expect for each, including negatives where nothing should fire. The negatives turned out to be where the collisions showed up: a skill that matches everything looks fine on recall, right up until I counted the turns it was taking from its neighbours.
Yes, and once skills become numerous, discovery becomes part of correctness. A good skill that never makes the active context is effectively unavailable. I think teams will need skill catalogs with budgets, tests, and pruning rules, not just more markdown.
"Discovery becomes part of correctness" is a great way to put it. The part I keep bumping into: pruning is the lever that reliably helps at scale, because every extra borderline description doesn't just cost budget — it adds one more wrong-trigger candidate. Curious what a skill catalog test would assert for you — trigger precision on a fixed prompt set, or something more structural?
I’d probably treat it like a search evaluation problem rather than a unit test.
Trigger precision and recall on a representative prompt corpus would be the baseline, but I’d also measure overlap. If adding or editing one skill significantly changes which other skills fire, that’s a regression even if the edited skill itself improves.
Another useful metric is trigger stability. Small wording changes to the same user intent shouldn’t completely change the selected skill. If they do, the descriptions are probably encoding lexical cues rather than semantic intent.
So I’d end up with three signals: precision/recall, overlap regression, and trigger stability. Together they tell you whether the skill catalog is becoming more predictable as it grows.
The overlap-regression signal is the one I'd have missed, and it's extra sneaky in this specific context: because the listing budget is shared, lengthening one skill's description can evict a different skill from what Claude sees at all. So skill B's triggering can regress without any semantic interaction between the two — pure budget displacement. Your trigger-stability point also doubles as a lint for descriptions: if paraphrasing the intent flips the selection, the description is keying on words, not the task. Have you tried running anything like that corpus in CI, or is this framework aspirational for you too?
I think this is a classic precision vs. recall problem. As the number of skills grows, optimizing each skill independently isn’t enough—you need to optimize the entire skill ecosystem. Sometimes deleting, merging, or narrowing a skill improves the overall trigger quality more than adding another description.
The precision/recall framing matches what I've seen — and merging is the underrated one of your three. In my case two skills with adjacent descriptions didn't just split recall, they kept competing for the same trigger and losing to each other; merging them made the combined skill fire more often than the pair had separately. Have you found a good heuristic for when to merge vs narrow?
A heuristic that has worked well for me is this:
Merge when two skills share the same user intent, differ mostly by implementation detail, and would usually require the same initial context. If choosing between them depends on information the user has not provided yet, they probably belong behind one router skill.
Narrow when the intent is genuinely different, but the wording overlaps. In that case, each description should include a clear positive trigger and a negative boundary: “Use this for X, not Y.”
I also look at correction patterns. If the model frequently chooses one skill and then has to switch to the other, merge them. If it triggers both for requests that should clearly belong to only one, narrow the boundaries.
So for me: repeated handoff suggests merge; repeated ambiguity suggests narrow.
That correction-pattern lens is the piece I was missing — repeated handoff vs repeated ambiguity gives you an observable signal to act on, instead of a design-time guess about intent. And the negative boundary in descriptions ("Use this for X, not Y") is underrated; I've seen very few skill authors state what a skill is not for, even though that's exactly what the model needs when two descriptions overlap. Definitely stealing both framings — thanks for taking the time to write this up properly.
good!