How Teams Manage Skills Files — /skill-doctor, Git, Evals

Every skill in your listing costs context on every turn whether or not the model ever uses it, so the management problem is not "where do I find more skills" but "how do I keep only the ones that pay for themselves". Two things converged on that answer this week. An Ask HN thread — "How do you manage skills files?", 239 points — produced a consistent practitioner pattern: skills in a Git repo, symlinked into each harness, scoped to your own repeatable workflows, tested with evals, pruned by deletion. And Claude Code 2.1.261, released 4 September 2026, shipped /skill-doctor, which lists which loaded skills have never been invoked and what each costs in context, then tells you where to turn them off. Put together, that is a lifecycle: store, load lazily, verify, measure, prune.

The five practices the thread agreed on

Read past the framework bikeshedding and the substantive answers in the Hacker News thread cluster tightly. First, version control with symlinks. Store skills in one Git repository and symlink them into each agent's folder across machines. One commenter's framing stuck: skills are "repeatable agent workflows written to your personal taste", updated when your workflow changes, not pulled from external collections. Second, scope to repeatable work. The leanest setups reported nine or ten project-specific skills — how to add logs, how to review code, how to cut a release — and an explicit "less is more" stance against downloading hundreds. Third, evals as integration tests. Several teams run skills through AI evals that assert on the output, and only write a skill when a problem recurs — "as a new problem arises, not just because". Fourth, drift detection. One team runs an agentic job every two weeks that diffs each skill against the official documentation it encodes and opens a PR when it has gone stale. Fifth, prune by deletion. Periodically remove skills and hooks, watch whether the model gets worse, and only restore the ones whose absence you felt.

Two smaller ideas are worth stealing. A "skill finder" skill: instead of preloading every skill summary, load one skill whose job is to search your Git repo for the relevant skill on demand. And post-session distillation: when the agent struggled through something, ask it to write what it learned into a skill file, then review before committing. That captures corrections, not just happy paths.

The one measured claim in the thread came from a commenter who reported skills cutting token output two to four times by removing the model's own self-prompting, and framed the organisational win as raising the floor — a consistent baseline across a team rather than a higher ceiling for one person. We have seen the same shape in our own transcripts.

What /skill-doctor adds

Practitioner discipline handles storage and testing. It does not tell you what each skill costs, because that number lives inside the harness. /skill-doctor does. Run it in an interactive Claude Code session (2.1.252 or later) and the report opens in the plugin manager's Stats tab: each loaded skill, whether it has ever been invoked, and its context cost. In a non-interactive -p run it prints as text, which means you can put it in a weekly cron and diff the output.

claude -p "/skill-doctor" > skill-report.txt
# then keep the last few reports and diff them:
diff skill-report.prev.txt skill-report.txt

The report flags skills that have never fired and says where to switch them off. That is the pruning signal the thread's "delete and see" method was approximating by feel. If a skill has not fired in a month of real sessions and costs a measurable slice of every turn, it goes.

Two related numbers to read alongside it. /cost now shows a per-session prompt-cache line and, since 2.1.260, a likely cause when the cache misses — tool definitions or the system prompt changed, or the session idled past the TTL. A skill listing that changes between turns is exactly the kind of system-prompt change that invalidates a cache, so a bloated or unstable skill set costs you twice: once in context and again in cache misses. We covered the cache-warm pattern in cache-warm subagent orchestration and the listing-budget setting in the skill listing budget fraction; both are worth rereading with /skill-doctor output in hand.

The lifecycle, as files

Store: one repo, one skill per folder

skills/
  add-structured-logs/SKILL.md
  review-pr/SKILL.md
  cut-release/SKILL.md
  evals/
    add-structured-logs.eval.md
    review-pr.eval.md
  install.sh

Each SKILL.md starts with frontmatter the harness reads — a name and a one-line description that states when it applies, because that description is what the model sees in the listing. A description that says "use when adding logs to a Node service; not for browser code" fires more precisely than "logging helper" and costs the same.

Load: symlink per harness, never copy

#!/usr/bin/env bash
# install.sh - link every skill into each harness that reads skills
set -euo pipefail
SRC="$HOME/dev/skills"
for target in "$HOME/.claude/skills" "$HOME/.codex/skills"; do
  mkdir -p "$target"
  for d in "$SRC"/*/; do
    name=$(basename "$d")
    ln -sfn "$d" "$target/$name"
  done
done

Copies drift; links do not. Teams using Guix Home or a dotfiles manager reported the same pattern with bidirectional links. If you distribute to a whole team, a plugin marketplace with version bumps is the heavier alternative — it gives you auto-updates at the cost of a publishing step.

Verify: an eval per skill

An eval is a fixed prompt plus an assertion. The simplest useful shape is a markdown file with the task, the files it should touch, and three things the output must contain. Run it on every change to the skill and once a fortnight regardless. When an eval fails after a model update, that is drift, and the fix is usually a sentence in the skill, not a rewrite.

Measure and prune: monthly

Run /skill-doctor, delete anything never invoked, move anything rarely invoked behind the skill-finder pattern so it loads on demand, and check /cost shows the cache staying warm across turns afterwards.

Where this is heading

The thread's opener noted that model capability may eventually absorb what skills encode. The counter-view, which we hold, is that skills are less about capability and more about your taste and your constraints — the release process your company uses, the log format your dashboards parse — and no base model learns those. What will change is the loading mechanism: lazy, searched, measured. /skill-doctor is the first harness feature built on that assumption.

An eval file, concretely

The thread agreed on evals but nobody posted one, so here is the smallest shape that has caught real regressions for us. One markdown file per skill, three parts: the fixture, the prompt, the assertions.

# evals/add-structured-logs.eval.md

## Fixture
repo: fixtures/node-service   (a 40-file Express app, committed)

## Prompt
Use the add-structured-logs skill to add request logging to src/routes/orders.ts

## Must be true after the run
- src/routes/orders.ts imports the shared logger, not console
- every new log line includes requestId and route
- no other file changed
- npm test passes

A runner script copies the fixture to a temp directory, runs Claude Code headless with the prompt, then checks each assertion with grep, git diff and the test command. Green means the skill still does what it says under the current model. Red after a model or harness update is drift, and the fix is nearly always one clarifying sentence in the skill's instructions.

The fortnightly drift job

The two-week documentation diff one commenter described is a small cron. Ours is a headless session with a fixed prompt: for each skill that encodes vendor documentation, fetch the current doc, compare it to the claims in the skill, and open a pull request with the changed lines and a link to the source. Running it against Claude Code's own changelog is what caught the /skill-doctor addition for this post. Schedule it, and add /skill-doctor output to the same report so usage and freshness arrive together.

# crontab: every other Monday, 07:00
0 7 */14 * 1 cd ~/dev/skills && claude -p "$(cat jobs/drift-check.prompt)" >> logs/drift.log 2>&1

One measurement closes the loop. After the first prune, read the /cost cache line across ten consecutive turns. If the skill listing is stable, the cache should report warm on every turn after the first. If it still misses, something else in the system prompt is churning — usually an MCP server re-announcing its tools on reconnect — and /skill-doctor was never going to catch that. Fix the churn first, then prune again, because a warm cache makes every remaining skill cheaper to carry.

The skill and rules files we run in production, including the evals layout above, ship in the Claude Code Production Pack; the CLAUDE.md Production Rules set is the companion for the always-loaded layer. If you want the architecture argument for what belongs in a skill versus a rule versus a subagent, our three-layer harness pattern post is the long version. Run /skill-doctor on your own setup today and delete the first thing it flags — then measure the next session with the token counter. Every product mentioned is available at wowhow.cloud — pay once, ship forever.

Originally published at wowhow.cloud