DEV Community

Cover image for Beyond Valid: What an Agent Needs From a SKILL.md
wolfejam.dev
wolfejam.dev Subscriber

Posted on Originally published at faf.one

Beyond Valid: What an Agent Needs From a SKILL.md

#agents #agentskills #tutorial #devtools

TL;DR: The Agent Skills validator tells you a SKILL.md is well formed. It can't tell you whether an agent will pick the skill at the right moment, run it safely, and know when to stop. This tutorial takes one skill from valid to useful in six steps, by hand, using only the Agent Skills specification and its reference validator.

New here? Agent Skills is an open format for giving AI agents new abilities: a folder with a SKILL.md file (YAML frontmatter, then Markdown instructions), plus any scripts or references it needs. It's an AAIF project, and many agent products read it.

The skill we'll fix

Here's a complete, valid skill:

---
name: release-notes
description: Helps with releases.
---

Write release notes.
Enter fullscreen mode Exit fullscreen mode

Check it with skills-ref, the specification's reference validator. With uv installed, no setup is needed:

uvx --from skills-ref agentskills validate release-notes
Enter fullscreen mode Exit fullscreen mode 
Valid skill: release-notes
Enter fullscreen mode Exit fullscreen mode

It passes. The specification's own description guidance calls Helps with PDFs. a poor example, and ours has the same problem. Valid is the floor, not the goal.

How an agent meets your skill

Two moments matter, and the specification describes both under progressive disclosure:

  1. Choosing. At startup the agent loads only each skill's name and description (about 100 tokens). That's all it has when deciding whether your skill fits the task.
  2. Doing. Once it picks your skill, it loads the whole body (under 5,000 tokens recommended) and follows it.

You can see the choosing view yourself:

uvx --from skills-ref agentskills to-prompt release-notes
Enter fullscreen mode Exit fullscreen mode 
<available_skills>
<skill>
<name>
release-notes
</name>
<description>
Helps with releases.
</description>
...
Enter fullscreen mode Exit fullscreen mode

That's the agent's whole view while it chooses. "Helps with releases" could mean tagging, publishing, versioning or notes. Steps 2 and 3 fix the choosing; steps 4 to 6 fix the doing.

Step 1: Make it valid

Run the validator on every skill, and fix every error before anything else. Here's what it catches:

---
name: Release_Notes
description: Helps with releases.
author: me
---
Enter fullscreen mode Exit fullscreen mode 
Validation failed for Release_Notes:
  - Unexpected fields in frontmatter: author. Only ['allowed-tools', 'compatibility', 'description', 'license', 'metadata', 'name'] are allowed.
  - Skill name 'Release_Notes' must be lowercase
  - Skill name 'Release_Notes' contains invalid characters. Only letters, digits, and hyphens are allowed.
Enter fullscreen mode Exit fullscreen mode

The usual fixes:

  • name: lowercase letters, digits and single hyphens, 64 characters at most, and the same as the folder's name.
  • description: 1 to 1,024 characters.
  • Extra fields (author, version, a client's own settings): move them under metadata, which takes any string keys.

Step 2: Say what it does, then when to use it

The specification says the description should cover both what the skill does and when to use it, with keywords that match real tasks. Put what it does first, in the third person (as the specification's good example does), then the trigger:

description: Drafts release notes for a new version from the git log and the pull requests merged since the previous tag. Use when the user asks for release notes, a changelog entry, or "what changed since v1.2".
Enter fullscreen mode Exit fullscreen mode

The trigger words are the ones a user would actually type: release notes, changelog, what changed.

Step 3: Say what it's not for (when a neighbour could trigger it)

If a nearby task could pick your skill by mistake, say so in the description. Release notes sit close to commit messages and pull request descriptions:

... Not for commit messages or pull request descriptions.
Enter fullscreen mode Exit fullscreen mode

Skip this when nothing nearby is easily confused. It's a guard, not a requirement.

Step 4: Say what it needs and what it touches

These are the facts an agent should have before step 1, not partway through:

**Needs:** git, and the GitHub CLI (`gh`) signed in to the repo.

**Touches:** reads tags, the git log and merged pull requests. Writes `RELEASE_NOTES.md` only. Publishes nothing: the user creates the release.
Enter fullscreen mode Exit fullscreen mode

Needs tells the agent to check its tools first, rather than failing at step 3. Touches tells it (and the person approving its actions) what will change: which files it writes, and whether anything leaves the machine. "Publishes nothing" matters as much as "publishes": it tells the agent where to stop and hand back.

The specification has an optional compatibility field for environment requirements, such as Requires git, docker, jq, and access to the internet. Use it if you like. It's one free-text line, so the reads, writes and publishing still belong in the body.

Step 5: Give the steps, in order

The specification recommends step-by-step instructions. Number them, and put the exact command in each one where there is one:

## Steps

1. Find the previous tag: `git describe --tags --abbrev=0`.
2. List the commits since it: `git log <tag>..HEAD --oneline`.
3. List the pull requests merged since it: `gh pr list --state merged --search "merged:>=<tag date>"`.
4. Group them under Added, Changed and Fixed, one line each, in words a user would use.
5. Write `RELEASE_NOTES.md` and show it to the user.
Enter fullscreen mode Exit fullscreen mode

Step 6: Say when it's done

An agent without a finish line keeps polishing, or stops early. Name the finished state of the work (a file, a result), not a step in the conversation:

**Done when:** `RELEASE_NOTES.md` lists every pull request merged since the previous tag, and the user has seen it.
Enter fullscreen mode Exit fullscreen mode

That's also a check the agent can run on itself before it reports back.

The result 

---
name: release-notes
description: Drafts release notes for a new version from the git log and the pull requests merged since the previous tag. Use when the user asks for release notes, a changelog entry, or "what changed since v1.2". Not for commit messages or pull request descriptions.
---

# Release notes

**Needs:** git, and the GitHub CLI (`gh`) signed in to the repo.

**Touches:** reads tags, the git log and merged pull requests. Writes `RELEASE_NOTES.md` only. Publishes nothing: the user creates the release.

## Steps

1. Find the previous tag: `git describe --tags --abbrev=0`.
2. List the commits since it: `git log <tag>..HEAD --oneline`.
3. List the pull requests merged since it: `gh pr list --state merged --search "merged:>=<tag date>"`.
4. Group them under Added, Changed and Fixed, one line each, in words a user would use.
5. Write `RELEASE_NOTES.md` and show it to the user.

**Done when:** `RELEASE_NOTES.md` lists every pull request merged since the previous tag, and the user has seen it.
Enter fullscreen mode Exit fullscreen mode

Still valid:

uvx --from skills-ref agentskills validate release-notes
Enter fullscreen mode Exit fullscreen mode 
Valid skill: release-notes
Enter fullscreen mode Exit fullscreen mode

Twenty lines. The specification asks for under 500 lines and about 5,000 tokens; if yours grows past that, move reference material into references/ and link it, one level deep, as the specification recommends. Then check those links point at files that exist.

The checklist

Run down this for any SKILL.md, yours or one you're about to install:

# Check Why the agent cares
1 skills-ref validate passes Clients may skip or reject an invalid skill
2 Description: what it does, then "Use when…" It's all the agent sees while choosing
3 "Not for…", if a neighbour could trigger it Stops the wrong skill being picked
4 Needs and Touches, before the steps Tools checked first; side effects known before they happen
5 Numbered steps The order is part of the instructions
6 Done when The agent knows when to stop and report
7 Under 500 lines; local links resolve The whole body loads at once; a missing file is a dead end

Check 1 is the specification's hard rules. Checks 2 to 7 come from its should guidance and recommendations, plus three facts an agent needs that the specification leaves to you: needs, touches and done.

Optional: the same facts as data

Steps 4 to 6 are prose, which suits a model reading the body. If you also want tools to read those facts, some authors put them in a small YAML block at the top of the body, called a FAF Block (also called a Context Block):

<!-- faf:start -->
```yaml
v: 1
needs: [git, "gh, signed in"]
reads: [tags, "git log", "merged pull requests"]
writes: [RELEASE_NOTES.md]
network: [github.com]
publishes: false
done_when: "RELEASE_NOTES.md lists every pull request merged since the previous tag, and the user has seen it"
```
<!-- faf:end -->
Enter fullscreen mode Exit fullscreen mode

It's valid in any skill today: the specification puts no restrictions on the body, and the body loads only after the skill is chosen, so the block costs nothing while the agent is choosing. Further reading on the format it comes from: faf.one/links.

Where to go deeper

Valid gets your skill loaded. The rest gets it chosen at the right moment, run safely, and finished.

Top comments (0)