DEV Community

weiwuji
weiwuji

Posted on

Agent Skills Are Not Documents — They Are Onboarding for Agents

#ai #agents #skills #engineering

The Pain: Agent Skills exploded across the ecosystem in a couple of weeks — Anthropic turned them into an open standard, Addy Osmani open-sourced agent-skills, tutorials are popping up everywhere. But after building skills myself, most people are doing it wrong: they cram pages of experience into one SKILL.md, and the agent either never finds it or cannot carry it in context when it does. The package lands in a directory and rots — three months later the model upgrades, tools change their interfaces, and the whole thing is obsolete.
What You'll Learn: Three engineering judgments for building skill packages: why a skill is not a document but onboarding; how Anthropic's progressive disclosure (metadata → SKILL.md → bundled files) actually saves context; and the real weak spot — the anti-rot maintenance loop. Every mechanism comes from a content-production agent system I have run for 276 days: a 60+ entry error ledger, nightly review, and 17 physical writing gates.

Opening: what an agent gets before it acts

In my previous article on coding-agent supply chains I said an agent's output is a proposal, not a finished product. Today I push one step further: the prepared materials an agent receives matter just as much. A skill package is not a document written for humans to read — it is an onboarding flow for the agent to walk through.

1. What Agent Skills really is: separate three things first

Anthropic's engineering blog post, Equipping Agents for the Real World with Agent Skills, makes it clear: one skill = one SKILL.md + optional bundled files in a conventional directory, and the agent discovers it and decides when to use it.

Many tutorials treat skills as "advanced prompts": lengthen the system prompt, add detail, add examples, wrap it in a shell called a skill. That is the first trap. Skills differ from prompts in three essential ways:

  • a prompt is pushed into context; a skill is pulled by the agent on demand;
  • a prompt has no boundary; a skill declares a description — the trigger condition — that states what it handles and what it does not;
  • changing a prompt re-runs everything; changing a skill only affects the tasks that hit it.

Building a skill for an agent is not writing documentation — it is writing onboarding. Documentation assumes people will read it voluntarily; onboarding assumes people will walk through it in order. An agent will never read anything proactively. It only opens SKILL.md after its description is matched.

2. Progressive disclosure: context carries an index, not the whole library

The easiest design detail to overlook — and the most valuable — is progressive disclosure, in three levels:

Three stacked cards showing the three levels of Agent Skills progressive disclosure. Level 1 metadata (blue, in context every turn) answers only when to use the skill — like routing knowing who a new hire is. Level 2 SKILL.md (teal, loaded on match) holds steps, boundaries, acceptance criteria — like a job manual you reach for when stuck. Level 3 bundled files (amber, called on use) keeps references and scripts on disk — like a mentor and environment that appear when real work starts. Teal conclusion bar: context carries an index, not the whole library — on-demand loading is how memory stays cheap

  • Level 1 — metadata: one description line, present in every turn. It only answers: when should this skill be used?
  • Level 2 — SKILL.md: loaded only when matched. Write steps, boundaries, acceptance criteria — short enough to finish in one read.
  • Level 3 — bundled files: references/ and scripts/ stay on disk and are called by path only when needed. Long material never eats context.

Context carries an index, not the whole library. This solves not "it does not fit" but "it cannot be found" — no matter how large the context window gets, you cannot stuff one hundred full skill packages into it. And at the exact moment the agent needs to decide, it usually only needs two or three pages.

This is how my own skill directory works: SKILL.md holds only trigger conditions, the main flow, and verification commands; references holds long specifications; scripts holds tools. Writing, diagramming, and review skills all follow this shape — after more than a year, none of them has ever blown up the context. A practical smell test: if your SKILL.md grows beyond one screen, you are writing a long prompt again.

My diagram skill just gained a new gate on September 8, 2026, which is a good example — all three figures for this article passed layout verification before use:

# Physical verification after generating diagrams (frozen 2026-09-08; real output from this article)
python3 /root/hermes-harness/verify/verify_image_layout.py \
  figs-as-20260908/as1-three-level.png \
  figs-as-20260908/as2-onboarding.png \
  figs-as-20260908/as3-rot-loop.png
# [PASS] as1-three-level.png: layout ok (1080x1350)
# [PASS] as2-onboarding.png: layout ok (1080x1350)
# [PASS] as3-rot-loop.png: layout ok (1080x1350)
Enter fullscreen mode Exit fullscreen mode

Behind it is the same logic: compile "should check" into "must pass a gate", instead of eyeballing every render. The gate code looks like this (real excerpt):

# Bottom conclusion-strip detection in verify_image_layout.py (real excerpt, 2026-09-08)
def _row_strip_ratio(px, w, y):
    tot = cnt = 0
    for x in range(0, w, 2):
        tot += 1
        if _is_strip(px[x, y]):
            cnt += 1
    return cnt / tot if tot else 0
Enter fullscreen mode Exit fullscreen mode

3. Why a skill is onboarding: move your new-hire playbook to the agent

Here is the judgment we settled on: building a skill for an agent and writing onboarding for a new hire are the same activity. Our team's three artifacts for onboarding — registration, job manual, mentor backup — map one-to-one onto a skill package:

Two-column mapping table between onboarding a new hire and building an agent skill. Row 1: registration form (role, team, when to call) maps to the description line. Row 2: job manual (SOP chapters) maps to SKILL.md. Row 3: mentor + environment maps to scripts + gates. Row 4: probation review (independent work means pass) maps to regression checks that block changes failing gates. Teal conclusion bar: a document waits to be read, onboarding walks you through it — agents only respond to the latter

  • Registration form → description: routing first learns who this is, which position they fill, and which tasks should call them;
  • Job manual → SKILL.md: SOP in chapters, reachable when something breaks, but not occupying the desk in normal times;
  • Mentor + environment → scripts + gates: they appear only when real work starts, and a gate stops deviation on the spot;
  • Probation review → regression checks: standing on your own counts as passing; a skill change that fails the gates never ships.

A document waits for someone to read it; onboarding walks someone through it — agents only respond to the latter. We figured this out while writing an onboarding-checklist for a new hire: writing a document is useless; you have to write "what to do first, what to do next, and how to verify when you are done." A skill that stores knowledge but no action order and no acceptance criteria leaves the agent unable to trust any single step even after opening it.

4. The real weak spot: skills rot — defense is a maintenance loop

Big labs open-source skills to demonstrate how to write them. Nobody teaches how to keep them alive. Skill rot is more common than failing to write one in the first place: a model release makes some steps outdated; a tool interface changes and the script fails on first run; a mistake you already hit never flows back, so the agent steps on it again next week.

My answer is a four-step maintenance loop that turns once every night:

Four-step anti-rot loop. Step 1 USE = PATCH (blue): fix what reads wrong every time a skill is used, debt never piles up. Step 2 NIGHTLY REVIEW (teal): scan the day's error ledger, find the common cause, log append-only (60+ entries). Step 3 FIX INTO THE SKILL (purple): sediment the correction into SKILL.md as a new section — remembering is not enough. Step 4 GATE REGRESSION (amber): skill changes run the 17 checks, broken versions are blocked before shipping. A return arrow below the row labels one full round every night. Teal conclusion bar: skills rot, a skill you maintain is a skill that keeps its value

# Nightly 21:00 self-evolution job (real cron: daily-self-evolution)
# 1. Scan the day's errors -> log into error-ledger (symptom / root cause / fix / status)
# 2. correction_logger extracts the lesson -> sediment into skill / SOP / gate
# 3. Mark status as embedded: remembering does not count, writing it into a mechanism does
Enter fullscreen mode Exit fullscreen mode

Step one is patch on use: every time a skill is used, if a phrase is inaccurate or a step redundant, fix it on the spot — never let it accumulate into debt. Step two is the nightly review: scan the day's error ledger and extract the common cause. Step three is fixing the correction into the skill: sediment it as a new section of SKILL.md — our error ledger has 60+ entries, append-only, re-fed every night. Step four is gate regression: every skill change runs the 17 checks; a broken version is stopped on the spot and never quietly ships as rot.

Skills rot. A skill you maintain is a skill that keeps its value. The worth of a skill package is not decided by how complete the first version is — it is decided by how many rounds of maintenance it survives.

Boundaries matter too: skills fit high-frequency, repeatable tasks whose acceptance can be coded — writing standards, diagram pipelines, review methods all qualify. For one-off tasks, or tasks still in exploration, a direct prompt is simpler. Building a skill for the sake of building one just creates a new kind of knowledge debt.

Closing

The open-sourcing wave will continue, standards will converge, tools will be replaced. What actually separates people is never how many skill packages they hold — it is who can keep their packages from going stale.

Building a skill for an agent is, at bottom, answering one question: do we want agents to be mentored like people, or parameterized like machines? My answer is the former — the people who write onboarding are the ones who truly understand what an agent needs.

One-liner: a skill package is not a document handed to the agent — it is onboarding — and its real value is decided by the maintenance loop that keeps it from rotting.

📖 Further reading from the Practitioner's series

About the author: Wu Ji (无记) — AI & digitalization practitioner focused on Agent engineering, Loop Engineering, and digital transformation. Practical, hands-on tutorials — follow along and it just works.

Top comments (0)